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(57) The picture coding method comprises: a sec- 
ond reference picture selection step of selecting, as a 
second reference picture, any one of coded pictures lo- 
cated after the current picture to be coded in display or- 
der (81 00); a predictive image generation step of spec- 
ifying a block in the first reference picture and a block in 
the second reference picture, and generating a predic- 
tive image from said two blocks through pixel interpola- 
tion (8102 and 8104); a coding step of coding a differ- 
ence between the predictive image and an image of a 
current block to be coded (S1 06 and SI 08); and an out- 
put step of outputting specification information for spec- 
ifying the second reference picture, together with a re- 
sult of the coding perfonned in the coding step. 
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Description 
Technical Field 

[0001] The present invention relates to a picture cod- 
ing method for coding a picture signal and a picture de- 
coding method for decoding the coded picture signal. 

Background Art 

[0002] With the development of multimedia applica- 
tions, it has become common to handle information of 
all sorts of media such as video, audio and text in an 
integrated manner. 

[0003] Against this backdrop, it becomes possible to 
handle media integrally by digitizing all types of media. 
[0004] However, since an enormous amount of data 
is contained in digitized pictures, a technique for com- 
pressing picture information is absolutely required to ac- 
cumulate or transmit such pictures. At the same time, it 
is also important to standardize compression tech- 
niques for the interoperability of compressed picture da- 
ta. The standard specifications on picture compression 
techniques include H.261 and H.263 standards recom- 
mended by ITU-T (International Telecommunication Un- 
ion-Telecommunication Standardization Sector) and 
MPEG (Moving Picture Experts Group)-1 , MPEG-2, and 
MPEG-4 of ISO. Currently, ITU is in an effort of stand- 
ardizing H.264 as the latest picture coding standard, 
which is called H.26L as a draft under standardization. 
[0005] Inter-picture prediction utilizing motion com- 
pensation is a technique commonly used In moving pic- 
ture coding methods such as MPEG-1 ,-2, and -4 and H. 
263. In motion compensation according to these moving 
picture coding methods, a picture in an input picture sig- 
nal is divided into rectangular areas (hereinafter referred 
to as "blocks") of a predetermined size, and predictive 
pixels are generated on a block-by-block basis from a 
motion vector indicating motion between pictures. 
[0006] tn order to Introduce Inter-plcture prediction 
using motion compensation, a description Is given of the 
following items (1)~(6): 

(1) Concept of B picture; 

(2) Interpolative prediction; 

(3) Picture number and reference index; 

(4) Direct mode; 

(5) Conventional picture coding apparatus; and 

(6) Conventional picture decoding sprats. 

(1) Concept of B picture 

[0007] Referring to FIG. 1 , a description is given of B 
pictures (bi-predictive pictures) according to H.26L. 
[0008] FIG. 1 is a conceptual diagram showing B pic- 
tures. 

[0009] A picture TP is the current B picture to be cod- 
ed, whereas each of pictures RP1 , RP2, RP3, and RP4 
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is an already coded picture. A block B1 Is a block on 
which inter-picture prediction has been performed with 
reference to a block RBI and a block RB2. A block B2 
is a block on which inter-picture prediction has been per- 
5 formed with reference to a block RB21 and a block 
RB22. And a block B3 is a block on which inter-picture 
prediction has been performed with reference to a block 
RB31 and a block RB32. 

[0010] As described above, the current picture TP, 
10 which is a B picture, includes blocks which shall be Inter- 
picture predicted with reference to respective blocks In- 
cluded in other two pictures. 

(2) Interpolative prediction 

15 

[0011] Referring to FIG. 2, a description Is given of 
Interpolative prediction. 

[0012] FIG. 2 Is a diagram explaining interpolative 
prediction. 

20 [0013] A block RBI and a block RB2 are two refer- 
ence blocks to be used for interpolative prediction, and 
a block PB indicates a predictive block obtained by in- 
terpolation. Here, a description is given, on the assump- 
tion that each block Is sized 4x4 pixels. 

25 [0014] XI (i) indicates a pixel value In the reference 
block RB1 , X2(i) indicates a pixel value In the reference 
block RB2, and P(i) indicates a pixel value in the predic- 
tive block PB. The pixel value P(i) can be obtained using 
a linear prediction equation such as below: 

30 

P(i)=A.X1(i)+B.X2(i)+C 

[0015] Here, A, B, and C are linear prediction coeffi- 

35 cients. The use of linear prediction coefficients depends 
on the case; only an average value (when A=1/2, B=1/2, 
and C=0) is used as linear predictive coefficients in the 
case of MPEG-1 and -2 (more strictly, a value to be ob- 
tained by rounding, to the closest integer value, the val- 

40 ue obtained by dividing X1 (i)+X2(i) to half), whereas in 
another case, other values are explicitly set as linear 
prediction coefficients. When other values are set as lin- 
ear prediction coefficients, such values are carried from 
a picture coding apparatus to a picture decoding appa- 

45 ratus in a coded picture signal. 

[001 6] Blocks on which inter-plcture prediction is per- 
formed through pixel Interpolation from plural reference 
pictures are referred to as "Interpolation predictive 
blocks". A B picture Is a picture capable of including In- 

50 terpolation predictive blocks inside. 

[0017] According to picture coding methods such as 
MPEG-1 and MPEG-2, an interpolation predictive block 
included in a B picture means an interpolation predictive 
block which is predicted from the reference picture RP1 

55 located before the current picture TP In display order 
and the reference picture RP3 located after the current 
picture TP In display order, as the block 81 shown In 
FIG. 1. 
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[001 8] In contrast, a recently proposed new B picture 
can further include an interpolation predictive block that 
refers to. as reference pictures, two pictures both of 
which are located either before or after the cun-ent pic- 
ture, as blocks B2 and 83 shown in FIG. 1. 
[0019] FIG. 3 is a diagram explaining an example 
case where two reference pictures of an interpolation 
predictive block are located, in display order, before the 
current picture to be coded that includes the interpola- 
tion predictive block. 

[0020] A picture P7 is the current picture to be coded. 
Each prcture Is coded in order of pictures P1 , P2, P3, 
P4, P5, P6, and P7. This means that the pictures PI, 
P2, P5, and P6 are coded before the current picture P7, 
and then stored into the multi-frame buffer (memory). 
[0021] An interpolation predictive block included in 
the current picture P7 is coded with reference to pixels 
in the picture PI and the picture P6. 
[0022] FIG. 4 is a diagram explaining an example 
case where two reference pictures of an interpolation 
predictive block are located, in display order, after the 
current picture to be coded that includes the interpola- 
tion predictive block. 

[0023] A picture P4 is the current picture to be coded. 
Each picture is coded in order of pictures PI , P2, P3, 
P4, and P5. This means that the pictures PI , P2, and 
P3 are coded before the current picture P4, and then 
stored into the multi-frame buffer. 
[0024] An interpolation predictive block included in 
the current picture P4 is coded with reference to pixels 
in the picture P2 and the picture P3. 
[0025] Note that a picture that can include blocks on 
which inter-picture prediction shall be performed from a 
single reference picture is referred to as a P picture, 
whereas a picture composed only of intra-picture pre- 
dictive blocks on which inter-picture prediction shall not 
be perfomried is referred to as an I picture. 
[0026] In H.26L, blocks in a B picture are coded using 
the maximum of two reference pictures. Here, in order 
to distinguish such two reference pictures, one of the 
reference pictures is referred to as a first reference pic- 
ture and the other a second reference picture. Moreover, 
motion vectors to be determined based on the first ref- 
erence picture and the second reference picture are re- 
spectively referred to as a first motion vector and a sec- 
ond motion vector. 

[0027] As for the current block B1 in FIG. 1 , for exam- 
ple, the picture RP1 is a first reference picture, the pic- 
ture RP3 is a second reference picture, the motion vec- 
tor MV1 is a first motion vector, and the motion vector 
MV2 is a second motion vector. Furthennore, prediction 
that is made based only on a first reference picture is 
referred to as a first reference picture prediction, where- 
as prediction that Is made based only on a second ref- 
erence picture is referred to as second reference pfcture 
prediction. 

[0028] Note that there is no need to make a distinction 
between first and second reference pictures and motion 



4 

vectors for a block on which Inter-picture prediction shall 
be made using one reference picture. However, for ex- 
planation purposes, the reference picture and motion 
vector of a block on which inter-picture prediction is per- 
5 formed using one reference picture are referred to re- 
spectively as a first reference picture and a first motion 
vector. 

(3) Picture number and reference index 

10 

[0029] FIGS. 5A and 5B are diagrams explaining pic- 
ture numbers and reference indices. 
[0030] Picture numbers and reference indices are in- 
formation used to uniquely identify reference pictures 
15 stored in the multi-frame buffer. In H,26L, a value which 
increments by one every time a picture is accumulated 
as a reference picture in the memory is assigned to such 
reference picture as Its picture number. 
[0031] Meanwhile, reference indices are used to 
specify reference pictures to be used for inter-picture 
prediction of the current picture to be coded. 
[0032] Reference indices are made up of first refer- 
ence indices for specifying a first reference picture and 
second reference Indices for specifying a second refer- 
ence picture. 

[0033] Referring to FIG. 5A, a description is given be- 
low of a method of assigning first reference indices and 
second reference indices. 

[0034] First, as first reference indices, values starting 
from 0 are assigned, in order of proximity to the current 
picture, to reference pictures with earlier display times 
than the current picture. After values starting from 0 are 
assigned to alt the reference pictures with earlier display 
times than the current picture, the subsequent values 
are assigned, In order of proximity to the current picture, 
to reference pictures with later display times than the 
current picture. 

[0035] Then, as second reference indices, values 
starting from 0 are assigned, in order of proximity to the 
cun^ent picture, to reference pictures with later display 
times than the current picture. After values starting from 
0 are assigned to all the reference pictures with later 
display times than the current picture, the subsequent 
values are assigned, in order of proximity to the current 
picture, to reference pictures with earlier display times 
than the current picture. 

[0036] For example, as shown In FIG. 5A, when 0 is 
indicated as a first reference Index, the first reference 
picture shall be a B picture with the picture number "14", 
whereas when 1 is indicated as a second reference in- 
dex, the second reference picture shall be a B picture 
with the picture number "13". 

[0037] In H.26L, it is possible to arbitrarily change the 
assignment of first reference indices and second refer- 
ence indices to reference pictures by explicitly indicating 
such change by use of a buffer control signal (RPSL 7 
within a header 1 shown in FIG. 8) included in coded 
picture signal. This change in the assignment of Indices 
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makes it possible for an arbitrary reference picture in- 
side the multi frame buffer to serve as a reference pic- 
ture with the second reference index value of 0, and as 
shown in FIG. 5B, for example, different values are as- 
signed as first reference indices and second reference 
indices for the respective picture numbers. 
[0038] Note that reference indices included in a block 
are represented as variable length code words, where 
a shorter code is assigned to a smaller value. 

(4) Direct mode 

[0039] FIG. 6 is a diagram explaining direct mode em- 
ployed by a conventional picture coding apparatus. 
[0040] Here, direct mode is a mode for perfomaing in- 
ter-picture prediction through pixel interpolation after 
determining reference pictures and motion vectors of 
the current bfock to be coded, using a method described 
below, based on motion vectors used at the time of cod- 
ing such reference pictures and based on pictures which 
were referred to at the time of coding such reference 
pictures. 

[0041 ] A picture TP is a B picture to be coded, where- 
as pictures RP1 , RP2, RP3. and RP4 are decoded pic- 
tures stored in the multi frame bufferfor use as reference 
pictures. Meanwhile, respective pictures indicated by 
dotted lines in FIG. 6 are non-reference pictures which 
shall not be stored in the multi frame buffer. 
[0042] Here, the first reference index RI1 of the pic- 
ture RP1 is 0, the first reference index RM of the picture 
RP2 is 1 , the second reference index RI2 of the picture 
RP3 is 0, and the second reference index RI2 of the pic- 
ture RP4 is 1. 

[0043] A block BO is a block to be coded in direct 
mode, and a block BOO is a block in the picture RP3 
which is co-located with the current block BO. Mean- 
while, a block RB01 is a reference block included in the 
picture RP1 , and a block RB02 is a reference block in- 
cluded in the reference picture RP3. A motion vector 
MVO, whk:h is a first motion vector to be used when cod- 
ing the block BOO, refers tothe picture RP1 . Such motion 
vector MVO shall be referred to as a scaling vector. 
[0044] The following equations are used to detennine 
a first motion vector MV01 and a second motion vector 
M V02 which are used for predicting the current block BO 
to be coded: 

MV01=TR1 XMVO/TRO 



MV02=-TR2X MVO/TRO 

[0045] In the above equations, the coefficients TR1 , 
TR2 and TRO are values used to determine motion vec- 
tors for direct mode, and an example of such values is 
the difference in display times between pictures. In FIG. 
6, TR1=2, TR2=1. andTR0=3. 



[0046] Assuming that an object that includes the cur- 
rent block moves in a constant manner within the dis- 
play, the first motion vector MV01 and the second mo- 
tion vector MV02 are determined by internally dividing 

5 the motion vector MVO by (i) the difference in display 
times between the current picture TP and the first refer- 
ence picture RP1 and (ii) the difference in display times 
between the current picture TP and the second refer- 
ence picture RP2. 

10 [0047] Note that not only the difference in display 
times but also the difference In the picture numbers as 
well as the number of pictures between pictures may 
also be used to detennine the first motion vector MV01 
and the second motion vector MV02. 

15 [0048] A set of the above coeffteients TR1 , TR2, and 
TRO shall be referred to as a direct mode scaling coef- 
ficient SP. 

[0049] In MPEG-4, since there is only one following 
reference picture at maximum in direct mode, a picture 

20 to be used as a second reference picture is uniquely de- 
termined. In H.26L, however, since more than one pic- 
ture can serve as a second reference picture, the picture 
with the second reference index RI2 of 0 shall be used 
as the second reference picture in direct mode. 

25 [0050] Moreover, in direct mode, a direct mode scal- 
ing coefficient SP transmitted for each picture is com- 
monly used by all the blocks included in each picture. 
Alternatively, scaling shall be performed in proportion to 
the difference in display times by use of display time in- 

30 formation of each picture. 

[0051] When FIG. 6 is to correspond to FIG. SA, the 
picture TP in FIG. 6 corresponds to a B picture located 
in the center (represented by the doted lines) in FIG. 5A, 
the picture RP3 in FIG. 6 con-esponds to a B picture with 

35 the picture number "15" in FIG. 5A, the picture RP4 in 
FIG. 6 corresponds to a B picture with the picture 
number "13" in FIG. 5A. the picture RP1 in FIG. 6 cor- 
responds to a B picture with the picture number "14" in 
FIG. 5A, and the picture RP2 in FIG. 6 corresponds to 

40 a 3 pfcture with the picture number "12" in FIG. 5A. 
Since non-reference pictures represented by doted 
liens such as the picture TP, shall not be referred to by 
other pictures, they shall not be stored in the multi frame 
buffer. Therefore, non-reference pictures are not be as- 

45 signed reference indices used for referring to such pic- 
tures, as the pictures shown in FIG. 5A are. 
[0052] Note that in FIG. 6, the first motion vector MVO 
is schematically illustrated as a vector that goes beyond 
space and time extending from the reference picture 

so RP3 to the reference picture RP1 . However, referring to 
FIG. 28, an actually stored first motion vector MVO indi- 
cates the amount of a locational change from the block 
BOO to the block B3. Therefore, the first motion vector 
MVO shown in FIG. 6 can be determined from a vector 

S5 extending from the block 600 to the block B3 as well as 
from the temporal distance between RP1 and RP3. 
[0053] In this specification, suppose that the motion 
vector MVO shown in FIG. 6 is actually stored for expla- 
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nation purposes. 

(5) Conventional picture coding apparatus 

[0054] Referring now to FIG. 7, a description is given 
of a conventional picture coding apparatus. 
[0055] FIG. 7 is a block diagram showing the config- 
uration of a conventional picture coding apparatus. The 
following gives a description of such picture coding ap- 
paratus. 

[0056] A picture coding apparatus 900 receives a pic- 
ture signal Img which is divided Into blocks, and per- 
forms processing on such signal on a block-by-block ba- 
sis. 

[0057] A subtracter 901 subtracts each of predictive 
image signals Pre from the picture signal Img, and out- 
puts residual signals Res. 

[0058] An image coding unit 902 obtains the residual 
signals Res so as to perform image coding processing 
such as DCT conversion and quantization on each of 
such residual signals Res, and outputs coded residual 
signals ER that include the quantized DCT coefficients 
and the like. 

[0059] An image decoding unit 904 obtains the coded 
residual signals ER so as to perfomn image decoding 
processing such as inverse quantization and inverse 
DCT conversion on each of such signals, and outputs 
decoded residual signals DR. 

[0060] An adder 905 adds each of the decoded resid- 
ual signals DR and each of the predictive image signals 
Pre, and outputs reconstructed image signals Rec. 
[0061] Reconstructed image signals Rec with the 
possibility to be referred to for inter-picture prediction of 
subsequent pictures, will be stored into a multi frame 
buffer 907. Since the multi frame buffer 907 has a limited 
amount of memory, picture data in the multi frame buffer 
907 which will not be used for later inter-picture predic- 
tion shall be removed from the multi frame buffer 907. 
[0062] A motion estimation unit 909 obtains a refer- 
ence picture RP stored in the multi frame buffer 907 so 
as to perfomi motion estimation on it, selects an opti- 
mum prediction type on a block-by-block basis from 
among the following prediction types: intra picture pre- 
diction, first reference picture prediction, second refer- 
ence picture prediction, and interpolatlve prediction us- 
ing a predetenmined method (a prediction type to be se- 
lected depends on a picture type). Then, the motion es- 
timation unit 909 outputs a first motion vector MV1 , a 
second motion vector MV2, a first reference Index RI1 , 
and a second reference index RI2 of the current block 
to be coded. 

[0063] An example method of selecting a prediction 
type employed by the motion estimation unit 909 is to 
select a prediction type that generates the smallest pre- 
diction error among all the prediction types. When intra 
picture prediction is selected as a prediction type, no 
motion vectors and reference Indices are outputted. 
When first reference picture prediction is selected, only 
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a first reference index and a first motion vector are out- 
putted. When second reference picture prediction is se- 
lected, only a second reference index and a second mo- 
tion vector are outputted. And when interpolatlve pre- 
5 diction is selected, a first reference index, a second ref- 
erence index, a first motion vector, and a second motion 
vector are outputted. 

[0064] As described above, in H.26L, a reference pic- 
ture with the second reference index rRI2 of 0 is selected 
10 as a second reference picture for direct mode. Accord- 
ingly, the second reference index rR12 with the value 0 
is inputted to a vector buffer 914 and a direct mode 
processing unit 910. 

[0065] Stored in the vector buffer 91 4 are scaling vec- 
15 tors rMV and the picture number of a picture refen^ed to 
by each of such scaling vectors rMV. Since a reference 
picture including a block that has been coded using a 
scaling vector rMV is a reference picture indicated by a 
second reference index rRI2, the vector buffer 914 re- 
ceives the second reference index rRI2 with the value 
0, and outputs the scaling vector rMV and the first ref- 
erence index rRII that indicates a picture referred to by 
the scaling vector rMV. 

[0066] The direct mode processing unit 910 receives 
a direct mode scaling coefficient SP, a scaling vector 
rMV, a first reference index rR1 1 , and a second reference 
index rRI2, and outputs a first motion vector sMVI , a 
second motion vector sMV2, a first reference index rRI1 , 
and a second reference index rRI2 for direct mode 
through the above-described direct mode processing. 
[0067] A prediction type selection unit 908 receives 
( i ) the picture signal Img, ( ii ) the reference pictures 
RP, ( ill ) the reference indices rRII and rRI2 and the 
motion vectors sMV1 and sMV2 which indicate the lo- 
cations of reference blocks for "direct mode", and ( iv ) 
the reference indices RI1 and RI2 and the motion vec- 
tors MV1 and MV2 which indicate the locations of refer- 
ence blocks used for prediction in "non-direct mode". 
Then, the prediction type selection unit 908 detemnines 
whether to use direct mode or not for block prediction, 
and outputs, to a variable length coding unit 903, type 
information PT indicating the determined prediction 
type. 

[0068] Here, the prediction type selection unit 908 se- 
lects a prediction type by selecting, for example, a small- 
er prediction error between prediction errors of input pix- 
els at the time of "direct mode" and "non-direct mode". 
[0069] Thus, direct mode is further included as a pre- 
diction type to be selected by the motion estimation unit 
909, in addition to intra picture prediction, first reference 
picture prediction, second reference picture prediction, 
and interpolatlve prediction other than in direct mode. 
[0070] Accordingly, when the selected prediction type 
indicates direct mode, a switch 911 switches to the "I" 
side, and the reference indices rRI1 and rRt2 as well as 
the motion vectors sMVI and sMV2 are used as the ref- 
erence Indies RI1 and RI2 as well as the motion vectors 
MV1 and MV2. 
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[0071 1 Meanwhile, when the prediction type indicates 
other than direct mode, the switch 911 switches to the 
"0" side. 

[0072] At the time of direct mode, the first motion vec- 
tor sMV1 used for coding a block in an already coded 
picture is used as a scaling vector. Moreover, a picture 
referred to by such first motion vector sMV1 is used as 
one of the two reference pictures in direct mode. There- 
fore, among the coded first reference Indices RI1 and 
first motion vectors MV1 , first reference Indices Rtl and 
first motion vectors MV1 with the possibility to be used 
in direct mode for subsequent pictures of the coded pic- 
ture are stored into the vector buffer 914. 
[0073] After a prediction type is detemnined, a first ref- 
erence index RI1 and a first motion vector MV1 are In- 
putted to the multl frame buffer 907, and a reference 
block RB1 corresponding to such inputted first reference 
Index RI1 and first motion vector MV1 is outputted from 
the multl frame buffer 907 to a pixel interpolation unit 
906. When a prediction type requires two reference 
blocks, a reference block RB2 corresponding to a sec- 
ond reference index RI2 and a second motion vector 
MV2 is outputted from the multi frame buffer 907 to the 
pixel interpolation unit 906. 

[0074] When Interpolatlve prediction Is selected, the 
pixel Interpolation unit 906 interpolates pixel values In 
positions in the two reference blocks RBI and RB2 cor- 
responding to each other, and outputs an interpolated 
block ReP. 

[0075] When the selected prediction type indicates in- 
terpolatlve prediction, a switch 912 switches to the "1" 
side, and uses the interpolated block ReP as a predic- 
tive Image signal Pre. 

[0076] When first reference picture prediction is se- 
lected, the multl frame buffer 907 outputs a reference 
block RB corresponding to the first reference index RI1 
and the first motion vector MV1 . When second reference 
picture prediction is selected, the multi frame buffer 907 
outputs a reference block RB corresponding to the sec- 
ond reference Index RI1 and the second motion vector 
M VI . Note that when Intra picture prediction is selected, 
the multl frame buffer 907 outputs a block RB that Is 
made up of pixels generated as a result of Intra picture 
prediction. 

[0077] Meanwhile, when other than interpolative pre- 
diction is indicated as a prediction type, the switch 
switches to the "0" side, and uses a reference block RB 
as a predictive image signal Pre. 
[0076] The variable length coding unit 903 perfonms 
variable length coding on the coded residual signal ER, 
the reference indices RI1 and RI2, the motion vectors 
MV1 and MV2, the direct mode scaling coefficient SP, 
and the type information PT, and incorporates the result- 
ant into a bit stream BSO so as to output it. 
[0079] FIG. 8 is a conceptual diagram showing the for- 
mat of the bit stream BSO. 

[0080] FIG. 8 shows the format of a part in the bit 
stream BSO where information equivalent to one picture 



is included. 

[0081] Such part includes the header 1 , a coded first 
block signal 2 of a block which was coded in direct mode, 
and a coded second block signal 3 of a block which was 
5 coded using interpolative prediction other than in direct 
mode. 

[0082] The coded second block signal 3 includes: a 
coded type signal 9 obtained by coding type information 
PT; a coded first Index signal 1 0 and a coded second 
10 index signal 1 1 obtained by coding reference indices RI1 
and RI2; and a coded MV1 signal 1 2 and a coded MV2 
signal 13 obtained by coding motion vectors MV1 and 
MV2. In the coded second block signal 3, the coded first 
index signal 10 and the coded second index signal 11 
15 as well as the coded MV1 signal 1 2 and coded MV2 sig- 
nal 13 are included in the bit stream BSO in the order 
shown In FIG. 8. 

[0083] Determination on whether to use a reference 
index RI1 or RI2 is made according to the coded type 
signal 9, and a first reference picture and a second ref- 
erence picture are determined by the data positions in 
the bit stream BSO of the coded first index signal 1 0 and 
the coded second index signal 11 . 
[0084] Meanwhile, the coded first block signal 2, 
which indicates that a block has been coded in direct 
mode, Includes the coded type signal 8 obtained by cod- 
ing the type infomaation PT, but does not include infor- 
mation indicating reference indices and motion vectors. 
[0085] The header 1 includes: first time difference in- 
formation 4, second time difference information 5, and 
time difference Information 6 obtained by coding the co- 
efficients TR1, TR2, and TRO shown in FIG. 6; and 
RPSL 7 that Indicates a change In the assignment of 
first reference indices or second reference indices to ref- 
erence pictures. Note, however, that when display time 
information of each picture is used for scaling a motion 
vector in direct mode, such information is not described 
in the header, and only display time Infomiation Is de- 
scribed in the header. 

(6) Conventional picture decoding apparatus 

[0086] FIG. 9 is a block diagram showing the config- 
uration of a conventional picture decoding apparatus. 
[0087] Here, a multi frame buffer 958, a pixel interpo- 
lation unit 957, a vector buffer 960, and a direct mode 
processing unit 954 equipped to a picture decoding ap- 
paratus 950 respectively have the same functions as 
those of the multl frame buffer 907, the pixel interpola- 
tion unit 906, the vector buffer 914, and the direct mode 
processing unit 91 0 of the picture coding apparatus 900 
shown in FIG. 7. A detailed description of them, there- 
fore, is omitted. Moreover, the same signals in FIG. 9 as 
those shown in FIG. 7 are assigned the same numbers, 
and no description Is given of them. 
[0086] The variable length decoding unit 951 receives 
the bit stream BSO so as to perform variable length de- 
coding on it, and outputs the coded residual signals ER, 
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the motion vectors MV1 and MV2, the reference Indices 
RI1 and RI2. the direct mode scaling coefficients SP (or 
the display time information of each picture), and the 
type infomiation PT. The image decoding unit 952 re- 
ceives the coded residual signals ER, perfomns inverse 
quantization and inverse DCT conversion on each of 
such residual signals ER, and outputs decoded residual 
signals DR. The adder 953 adds each of the decoded 
residual signals DR and each of the predictive image 
signals Pre, and outputs decoded picture signals Dim 
to outside the picture decoding apparatus 950. The multi 
frame buffer 958 stores some of the decoded picture sig- 
nals Dim necessary for inter-picture prediction. 
[0089] The vector buffer 960 holds a scaling vector 
rMV and information used for identifying a picture refe- 
reed to by such scaling vector rMV (the first reference 
index rRI1 ). Moreover, the vector buffer 980 receives the 
second reference Index rRI2 with the value 0, and out- 
puts the scaling vector rMV and the first reference index 
rRI1. 

[0090] The direct mode processing unit 954 perfonns 
the same processing as that of the direct mode process- 
ing unit 910 shown in FIG. 7. 

[0091] When the prediction type is other than direct 
mode, the switch 955 switches to the "0" side. Then, the 
multi frame buffer 958 obtains the reference indices RI1 , 
and RI2 as well as the motion vectors MV1 and MV2. 
[0092] When the prediction type indicates direct 
mode, the switch 955 switches to the "1 " side. Then, the 
multi frame buffer 958 obtains the reference indices rRM 
and rRt2, as well as the motion vectors sMVI and sMV2 
as the reference indices R11 and RI2 as well as the mo- 
tion vectors MV1 and MV2. 

[0093] When the selected prediction type indicates in- 
terpolative prediction, the multi frame buffer 958 outputs 
a reference block RBI corresponding to the first refer- 
ence index RI1 and the first motion vector MV1 as well 
as a reference block RB2 corresponding to the second 
reference index RI2 and the second motion vector M V2. 
Subsequently, the pixel interpolation unit 957 interpo- 
lates pixel values corresponding to the respective two 
reference blocks RBI and RB2, and outputs an interpo- 
lated block ReP. 

[0094] When first reference picture prediction is se- 
lected, the multi frame buffer 958 outputs a reference 
block RB corresponding to the first reference index RI1 
and the first motion vector MV1 . When second reference 
picture prediction is selected, the multi frame buffer 958 
outputs a reference block RB corresponding to the sec- 
ond reference index RI2 and the second motion vector 
M V2. Note that when intra picture prediction is selected, 
the multi frame buffer 958 outputs a block RB that is 
made up of pixels generated as a result of intra picture 
prediction. 

[0095] Meanwhile, when interpolative prediction is in- 
dicated as a prediction type, the switch 956 switches to 
the "0" side, and the interpolated block ReP is used as 
a predictive Image signal Pre, 
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[0096] Meanwhile, when other than interpolative pre- 
diction is indicated as a prediction type, the switch 956 
switches to the "1 " side, and a reference block RB is 
used as a predictive image signal Pre. 
5 [0097] Amongst the decoded first reference indices 
RI1 and first motion vectors MV1 , first reference indices 
RI1 and first motion vectors MV1 with the possibility to 
be used in direct mode for subsequent pictures of a de- 
coded picture are stored into the vector buffer 960. 
10 [0098] The picture decoding apparatus 950 with the 
above configuration decodes the bit stream BSO 
through the above-described processing, and outputs 
the resultant as decoded picture signals Dim, 
[0099] Here, in the above-described conventional pic- 
^5 ture coding apparatus 900 and picture decoding appa- 
ratus 950, a B picture is coded with reference to ( i ) two 
forward pictures, ( ii ) two backward pictures, or ( iii ) one 
forward picture and one backward picture. Regarding 
general video, however, a higher coding efficiency can 
be achieved by referring to one forward picture and one 
backward picture rather than referring to two forward 
pictures or two backward pictures, since highly accurate 
interpolative prediction is possible in the former case. 
[0100] However, in the above-described conventional 
picture coding apparatus 900 and picture decoding ap- 
paratus 950, since the value 0 of a second reference 
index rRI2 is assigned to an arbitrary reference picture 
in the multi frame buffers 907 and 908 as described with 
reference to FIG, 5B. Therefore, in direct mode, a B pic- 
ture is coded with reference to two forward pictures or 
two backward pictures in many cases, whteh causes a 
problem that coding efficiency in coding of B pictures is 
lowered. 

[0101] Also, as shown in FIG. 6, in direct mode, the 
current picture TP is coded by use of the first reference 
picture RP1 and the second reference picture RP3 as 
well as the scaling vectors MVO (the first motion vectors 
MVO). Here, the second reference picture is a picture 
whose second reference index RI2 value is 0, and the 
first reference picture is a picture which is referred to by 
the first motion vector MVO of the block BOO in the sec- 
ond reference picture. 

[0102] Therefore, in order to realize direct mode, the 
picture coding apparatus 900 is required to hold the first 
motion vectors MVO used for coding the second refer- 
ence picture RP3 as well as information used to identify 
the picture RP1 referred to by each of such motion vec- 
tors MVO. 

[0103] By the same token, the picture decoding appa- 
ratus 950 is also required to hold the first motion vectors 
MVO of a decoded picture as well as information used 
to identify the picture RP1 referred to by each of such 
motion vectors MVO. 

[0104] In H.263, a motion vector can be switched to 
another one on a 4x4 block basis at the minimum and 
a reference picture can be switched to another one on 
an 8x8 block basis at the minimum. Therefore, in the 
case of a QCIF-srzed image (176x144 blocks), the 
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memory is required to be capable of holding information 
for identifying 1584 motion vectors and 396 reference 
pictures per reference picture. 

[0105] Moreover, in H.26L, since a change can be 
freely made in the assignment of reference indices to 
reference pictures, any reference picture in the multi 
frame buffer can be a reference picture con-esponding 
to the second reference index RI2=0. This requires the 
picture coding apparatus 900 and the picture decoding 
apparatus 950 to hold the first motion vectors of all ref- 
erence pictures stored in the multi frame buffers 907 and 
958 as well as information used to identify pictures re- 
ferred to by such first motion vectors. 
[01 06] Since the amount of memory required to store 
such first motion vectors and picture identification infor- 
mation is proportional to the number of reference pic- 
tures, the vector buffers 914 and 960 that hold such in- 
formation are required to have vast storage capacity. In 
other words, since an enormous amount of information 
is to be stored in the vector buffers 91 4 and 960, load is 
inevitably placed on the inputting/outputting of such in- 
fomnation. As a result, coding processing becomes com- 
plicated, leading to a decrease in coding efficiency. 
[01 07] It is an object of the preset invention to provide 
a picture coding method and a picture decoding method 
capable of offering an improved coding efficiency. 

Disclosure of Invention 

[0108] In order to achieve the above object, the pic- 
ture coding method according to the present invention 
is a picture coding method for coding a current picture 
to be coded on a block-by-block basis with reference to 
already coded pictures, comprising: a second reference 
picture selection step of selecting, as a second refer- 
ence picture, any one of coded pictures located after the 
current picture in display order; a first reference picture 
specification step of specifying, as a first reference pic- 
ture, any one of coded pictures based on a motion vector 
used for coding a predetemiined block in the second ref- 
erence picture; a predictive image generation step of 
specifying a block in the first reference picture and a 
block in the second reference picture based on the mo- 
tion vector, and generating a predictive image from said 
two blocks through pixel interpolation; a coding step of 
coding a difference between the predictive image and 
an image of a current block to be coded; and an output 
step of outputting specification information for specify- 
ing the second reference picture, together with a result 
of the coding performed in the coding step. 
[0109] Here, in the output step, the specification infor- 
mation that includes a second reference index indicating 
the second reference picture may be outputted. Moreo- 
ver, in the output step, information indicating a method 
for changing second reference indices may be outputted 
as the specification information, said second reference 
indices being assigned to the respective coded pictures 
in order to identify said coded pictures. 
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[0110] Accordingly, when remapping occurs, since a 
coded picture located after the current picture in display 
order is selected as the second reference picture, it be- 
comes possible to prevent two coded pictures located 
5 before the current picture in display order from being ref- 
ereed to as in the conventional cases, and therefore to 
improve the probability that two coded pictures located 
before and after the current picture in display order are 
refereed to. This consequently Improves coding efficien- 
10 cy. Furthennore, since the specification information is 
outputted, it is possible to correctly decode a result of 
the coding executed in the coding step, by selecting the 
same picture as the second reference picture selected 
at the time of coding, based on such specification infor- 
ms mation. 

[0111] Also, the picture coding method according to 
the present invention is a picture coding method for cod- 
ing a current picture to be coded on a block-by-block 
basis with reference to already coded pictures, compris- 
ing: a second reference picture selection step of select- 
ing, as a second reference picture, a coded picture from 
among coded pictures that are located after the current 
picture in display order, said coded picture being as- 
signed a smallest second reference index of all second 
reference indices assigned to the respective coded pic- 
tures in order to identify said coded pictures; a first ref- 
erence picture specification step of specifying, as a first 
reference picture, any one of coded pictures based on 
a motion vector used for coding a predetermined block 
in the second reference picture; a predictive image gen- 
eration step of specifying a block in the first reference 
picture and a block in the second reference picture 
based on the motion vector, and generating a predictive 
image from said two blocks through pixel interpolation; 
and a coding step of coding a difference between the 
predictive image and an image of a current block to be 
coded. 

[01 1 2] Here, in the second reference picture selection 
step, the coded picture with the smallest second refer- 
ence index may be specified according to assignment 
Information indicating how the second reference indices 
are assigned, and said specified coded picture may be 
selected as the second reference picture. 
[0113] Accordingly, when remapping occurs, since a 
coded picture located after the current picture in display 
order is selected as the second reference picture, it be- 
comes possible to prevent two coded pictures located 
before the current picture in display order from being ref- 
ereed to as In the conventional cases, and therefore to 
improve the probability that two coded pictures located 
before and after the current picture in display order are 
refereed to. This consequently improves coding efficien- 
cy. Furthermore, since the picture with the smallest ref- 
erence index is selected as the second reference pic- 
ture, it is possible to conrectty decode a result of the cod- 
ing perfonned in the coding step, by uniquely selecting 
the same picture as the second reference picture select- 
ed at the time of coding. 
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[0114] Here, the picture coding method according to 
the present invention is a picture coding method for cod- 
ing a current picture to be coded on a bloclc-by-bloci< 
basis with reference to already coded pictures, compris- 
ing: an assignment step of assigning, to respective cod- 
ed pictures, second reference indices for identifying said 
coded pictures in a manner in which a second reference 
index indicating 0 is assigned to any one of coded pic- 
tures located after the current picture in display order; a 
re-assignment step, In which it Is possible to re-assign 
the second reference indices in a manner different from 
the manner used in the assignment step, of assigning 
the second reference index indicating 0 only to any one 
of the coded pictures located after the current picture in 
display order, when re-assigning the second reference 
indices; a second reference picture selection step of se- 
lecting, as a second reference picture, the coded picture 
assigned the second reference Index Indicating 0 from 
among the coded pictures located after the current pic- 
ture in display order; a first reference picture specifica- 
tion step of specifying, as a first reference picture, any 
one of the coded pictures based on a motion vector used 
for coding a predetermined block in the second refer- 
ence picture; a predictive Image generation step of 
specifying a block In the first reference picture and a 
block in the second reference picture based on the mo- 
tion vector, and generating a predictive image from said 
two blocks through pixel interpolation; and a coding step 
of coding a difference between the predictive image and 
an image of a current block to be coded. 
[01 15] Here, in the re-assignment step, it may be pro- 
hibited to change the second reference Index of the cod- 
ed picture which Is assigned the second reference Index 
indicating 0 in the assignment step. 
[01 16] Accordingly, in the assignment step and the re- 
assignment step, since the second reference index in- 
dicating 0 is assigned to a coded picture that is located 
after the current picture to be coded in display order, a 
coded picture that is located after the current picture to 
be coded in display order Is selected as the second ref- 
erence picture. As a result, when remapping occurs, 
since a coded picture located after the current picture in 
display order is selected as the second reference pic- 
ture, it becomes possible to prevent two coded pictures 
located before the current picture in display order from 
being refereed to as in the conventional cases, and 
therefore to improve the probability that two coded pic- 
tures located before and after the current picture in dis- 
play order are refereed to. This consequently Improves 
coding efficiency. 

[0117] Furthermore, the picture coding method ac- 
cording to the present invention is a picture coding meth- 
od for coding a current picture to be coded on a block- 
by-biock basis with reference to already coded pictures, 
comprising: an assignment step of assigning, to respec- 
tive coded pictures, first reference indices for Identifying 
said coded pictures; a re-assignment step of re-asslgn- 
Ing the first reference indices to predetemrilned coded 



pictures or slices In a manner different from the manner 
used In the assignment step; a first reference picture se- 
lection step of selecting, as a first reference picture, a 
coded picture assigned a first reference index indicating 

5 a predetermined value, from among coded pictures lo- 
cated before the current picture in display order; a pre- 
dictive image generation step of generating a predictive 
image through pixel Interpolation based on a predeter- 
mined block In the first reference picture; and a coding 

10 step of coding a difference between the predictive image 
and an image of a current block to be coded, wherein in 
the re-assignment step, the first reference index indicat- 
ing said predetermined value is assigned preferentially 
to the coded picture which is referred to for a long period 

IS of time. 

[0118] Accordingly, when a scene change occurs, 
since remapping is performed In a manner in which the 
first reference index indicating 0 is preferentially as- 
signed to a coded picture in a memory that stores coded 

^0 pictures to be referred to for a long period of time, and 
the coded picture with such first reference index of 0 is 
selected as the first reference picture, it is possible to 
improve the probability that a coded picture similar to 
the current picture to be coded Is referred to. This con- 

25 sequently improves coding efficiency. 

[0119] Moreover, the picture coding method accord- 
ing to the present invention is a picture coding method 
for coding a current picture to be coded on a block-by- 
block basis with reference to already coded pictures, 

30 comprising: an assignment step of assigning, to respec- 
tive coded pictures, second reference indices for Iden- 
tifying said coded pictures; a second reference picture 
selection step of selecting, as a second reference pic- 
ture, a coded picture which is assigned a second refer- 

35 ence index indicating 0, from among coded pictures lo- 
cated after the current picture in display order; a first 
reference picture specification step of specifying, as a 
first reference picture, any one of the coded pictures 
based on a motion vector used for coding a predeter- 

40 mined block in the second reference picture; a predictive 
image generation step of specifying a block in the first 
reference picture and a block in the second reference 
picture based on the motion vector, and generating a 
predictive image from said two blocks through pixel in- 

^5 terpolation; and a coding step of coding a difference be- 
tween the predictive image and an image of a current 
block to be coded, wherein in the assignment step, the 
second reference index indicating 0 Is prohibited from 
being assigned to a coded picture that has been located 

50 before the current picture in display order. 

[0120] Accordingly, since the second reference index 
indicating 0 is prohibited from being assigned to a coded 
picture that has been located before the current picture 
to be coded in display order, such coded picture that has 

55 been located after the current picture In display order 
shall never be selected as the second reference picture. 
As a result, It becomes possible to omit the processing 
for unnecessarily storing motion vectors used for coding 
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the blocks on such coded pictures, which consequently 
leads to innproved coding efficiency. 
[0121] Here, the picture coding method with the 
above configuration may further comprise a deletion 
step of deleting, from a memory storing motion vectors 
used forcoding the coded pictures, a motion vector used 
for coding the coded picture that has been located be- 
fore the current picture in display order. 
[01 22] Accordingly, it becomes possible to reduce the 
amount of memory by preventing motion vectors unnec- 
essary for coding from being stored into the memory. 
[0123] Also, the picture coding method according to 
the present invention is a picture coding method forcod- 
ing a current picture to be coded on a bloc)(-by-bloclc 
basis with reference to already coded pictures, compris- 
ing: a selection step of selecting, from among a plurality 
of coded pictures stored in a storage unit, a coded pic- 
ture corresponding to a first reference index as a first 
reference picture and a coded picture corresponding to 
a second reference index as a second reference picture, 
by use of said first reference index and said second ref- 
erence index assigned to said plurality of coded pic- 
tures; a predictive image generation step of generating 
a predictive image from a block in the first reference pic- 
ture and a block in the second reference picture through 
pixel interpolation; and an output step of coding a pre- 
diction enror which is a difference between the current 
picture and the predictive image, and outputting a coded 
picture signal that includes a signal obtained by coding 
said prediction error, wherein in the selection step, the 
coded picture which satisfies the following condition is 
selected as the first reference picture from among the 
coded pictures: a coded picture which Is located before 
the current picture In display order and which is as- 
signed a smallest reference Index, and in the predictive 
image generation step, ( i ) a second motion vector of a 
block which refers to the first reference picture is calcu- 
lated, based on a first motion vector used to code said 
block which is in the second reference picture and co- 
located with a current block to be coded In the current 
picture, ( ii ) a third motion vector of the current block 
which refers to the first reference picture Is calculated 
and a fourth motion vector of the current block which 
refers to the second reference picture is calculated, both 
based on said second motion vector, and { iii ) the pre- 
dictive image is generated through pixel interpolation 
from a block in the first reference picture referred to by 
the third motion vector and a block in the second refer- 
ence picture referred to by the fourth motion vector. 
[01 24] Accordingly, since the same first reference pic- 
ture Is commonly used for each block in the current pic- 
ture to be coded regardless of which picture a first mo- 
tion vector of the second reference picture refers to, it 
becomes unnecessary to store information for identify- 
ing the first reference picture Into the memory, as In the 
conventional cases. This can reduce the amount of 
memory as well as simplifying coding processing so as 
to improve coding efficiency. 



[0125] Moreover, the picture decoding method ac- 
cording to the present Invention Is a picture decoding 
method for decoding a coded picture on a block-by- 
btock basis with reference to already decoded pictures, 
5 comprising: an obtainment step of obtaining specifica- 
tion infomriation and a coded signal, the specification in- 
formation being used for specifying, as a second refer- 
ence picture, a decoded picture to be referred to for de- 
coding a cun-ent picture to be decoded and the coded 
10 signal being related to said current picture; a second ref- 
erence picture selection step of selecting the second ref- 
erence picture from among decoded pictures that are 
located after the current picture in display order based 
on the specification information; a first reference picture 
IS specification step of specifying, as a first reference pic- 
ture, any one of decoded pictures based on a motion 
vector used for decoding a predetenmined block in the 
second reference picture; a predictive image generation 
step of specifying a block in the first reference picture 
and a block in the second reference picture based on 
the motion vector, and generating a predictive Image 
from said two blocks through pixel interpolation; and a 
decoding step of decoding a current block to be decoded 
based on the predictive image and the coded signal. 
[01 26] Here, in the obtainment step, the specification 
information that includes a second reference index indi- 
cating the second reference picture may be obtained, 
and In the second reference picture selection step, the 
second reference picture indicated by said second ref- 
erence index may be selected. Furthermore, in the ob- 
tainment step, the specification Information that indi- 
cates a method for changing second reference Indices 
may be obtained, said second reference indices being 
assigned to the respective decoded pictures in order to 
identify said decoded pictures, and in the second refer- 
ence picture selection step, the decoded picture Indicat- 
ed by said method for changing the second reference 
indices may be selected as the second reference pic- 
ture. 

[0127] Accordingly, It becomes possible to correctly 
decode a pbture that has been decoded using the pic- 
ture coding method according to the present Invention, 
by uniquely selecting the second reference picture 
based on the specification information. 
[0128] In addition to the above, the picture coding 
method according to the present invention may be a pic- 
ture coding method, comprising: a first step of selecting 
either at least a first or second reference frame from 
among a plurality of coded frames stored in a storage 
unit by use of a first relative index and a second relative 
index assigned to said coded frames, in order to select 
a first reference frame and a second reference frame to 
be referred to when obtaining a block on a current frame 
to be coded by means of motion compensation; a sec- 
ond step of generating, through pixel interpolation, a 
predictive Image from a block on at least either the first 
or second reference frame obtained by means of motion 
compensation; and a third step of coding a prediction 
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error which is a difference between the Input cun-ent 
frame and the predictive Image, and outputting a coded 
picture signal that includes a signal obtained by coding 
said prediction en-or, wherein In the first step, a refer- 
ence frame which satisfies the following condition Is se- 5 
lected as the second reference frame from among the 
coded frames: a reference frame which is located after 
the current frame in display order and which is assigned 
a smallest second relative index, and in the second step. 
( I )a frame is regarded as the first reference frame, said io 
frame being refereed to by a motion vector used for mo- 
tion-compensating a block which is in the second refer- 
ence frame and co-located with a predetemiined block 
In the current frame, ( il ) based on said motion vector, 
a motion vector to thie first reference frame and a motion is 
vector to the second reference frame are calculated us- 
ing a predetermined method, and ( lil ) the predictive im- 
age is generated through pixel interpolation from a block 
obtained from the motion vector to the first reference 
frame and a block obtained from the motion vector to 20 
the second reference frame. 

[0129] In addition to the above, the picture decoding 
method according to the present invention may be a pic- 
ture decoding method, comprising: a first step of obtain- 
ing a coded picture signal that includes a signal obtained 25 
by coding a prediction error; a second step of selecting 
either at least a first or second reference frame from 
among a plurality of decoded frames stored in a storage 
unit by use of a first relative index and a second relative 
index assigned to said decoded frames, in order to se- 30 
lect a first reference frame and a second reference 
frame to be referred to when obtaining a block on a cur- 
rent frame to be decoded by means of motion compen- 
sation; a third step of generating, through pixel interpo- 
lation, a predictive image from a block on at least either 35 
the first or second reference frame obtained by means 
of motion compensation; a fourth step of generating a 
decoded picture of a frame from the predictive image 
and the decoded prediction error; and a fifth step of stor- 
ing, in the storage unit, a decoded picture of a frame 40 
with a possibility to be used for Inter-frame prediction, 
wherein in the second step, a reference frame which sat- 
isfies the following condition is selected as the second 
reference frame from among the decoded frames: a ref- 
erence frame which is located after the current frame in 
display order and which is assigned a smallest second 
relative index, and in the third step, ( i ) a frame is re- 
garded as the first reference frame, said frame being 
refereed to by a motion vector used for motion-compen- 
sating a block which is in the second reference frame so 
and co-located with a predetermined block in the current 
frame, ( ii ) based on said motion vector, a motion vector 
to the first reference frame and a motion vector to the 
second reference frame are calculated using a prede- 
termined method, and ( lil ) the predictive image is gen- ss 
erated through pixel interpolation from a block obtained 
from the motion vector to the first reference frame and 
a block obtained from the motion vector to the second 



reference frame. 

[0130] In addition to the above, the picture coding 
method according to the present invention may be a pic- 
ture coding method, comprising: a first step of selecting 
either at least a first or second reference frame from 
among a plurality of coded frames stored in a storage 
unit by use of a first relative index and a second relative 
index assigned to said coded frames. In order to select 
a first reference frame and a second reference frame to 
be referred to when obtaining a block on a cun-ent frame 
to be coded by means of motion compensation; a sec- 
ond step of generating, through pixel Interpolation, a 
predictive image from a block on at least either the first 
or second reference frame obtained by means of motion 
compensation; and a third step of coding a prediction 
error which is a difference between the input current 
frame and the predictive Image, and outputting a coded 
picture signal that includes a signal obtained by coding 
said prediction error, wherein in the first step, a frame 
which Is located after the current frame in display order 
is selected as the second reference frame, from among 
the coded frames, in the second step, ( i )a frame is re- 
garded as the first reference frame, said frame being 
refereed to by a motion vector used for motion-compen- 
sating a block which is In the second reference frame 
and co-located with a predetermined block In the current 
frame, ( ii ) based on said motion vector, a motion vector 
to the first reference frame and a motion vector to the 
second reference frame are calculated using a prede- 
termined method, and ( iii ) the predictive image is gen- 
erated through pixel interpolation from a block obtained 
from the motion vector to the first reference frame and 
a block obtained from the motion vector to the second 
reference frame, and in the third step, the second rela- 
tive index indicating the second reference frame select- 
ed in the first step is Incorporated Into the coded picture 
signal. 

[0131] In addition to the above, the picture decoding 
method according to the present invention may be a pic- 
ture decoding method, comprising: a first step of obtain- 
ing a coded picture signal that includes a signal obtained 
by coding a prediction error; a second step of selecting 
either at least a first or second reference frame from 
among a plurality of decoded frames stored in a storage 
unit by use of a first relative index and a second relative 
index assigned to said decoded frames, in order to se- 
lect a first reference frame and a second reference 
frame to be referred to when obtaining a block on a cur- 
rent frame to be decoded by means of motion compen- 
sation; third step of generating, through pixel interpola- 
tion, a predictive image from a block on at least either 
the first or second reference frame obtained by means 
of motion compensation; a fourth step of generating a 
decoded picture of a frame from the predictive image 
and the decoded prediction error; and a fifth step of stor- 
ing. In the storage unit, a decoded picture of a frame 
with a possibility to be used for inter-frame prediction, 
wherein In the first step, the second relative index is ob- 
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tained from the coded picture signal Including second 
relative indices, in the second step, a frame indicated 
by the second relative index obtained in the first step is 
regarded as the second reference frame, in the third 
step, ( i ) a frame is regarded as the first reference 
frame, said frame being refereed to by a motion vector 
used for motion-compensating a block which is in the 
second reference frame and co-located with a predeter- 
mined block in the cun^ent frame, ( ii ) based on said mo- 
tion vector, a motion vector to the first reference frame 
and a motion vector to the second reference frame are 
calculated using a predetermined method, and ( iii ) the 
predictive image is generated through pixel interpolation 
from a block obtained from the motion vector to the first 
reference frame and a block obtained from the motion 
vector to the second reference frame. 
[0132] In addition to the above, the picture coding 
method according to the present invention may be a pic- 
ture coding method, comprising: a first step of selecting 
either at least a first or second reference frame from 
among a plurality of coded frames stored in a storage 
unit by use of a first relative index and a second relative 
index assigned to said coded frames, in order to select 
a first reference frame and a second reference frame to 
be referred to when obtaining a block on a current frame 
to be coded by means of motion compensation; a sec- 
ond step of generating, through pixel interpolation, a 
predictive image from a block on at least either the first 
or second reference frame obtained by means of motion 
compensation; and a third step of coding a prediction 
error which is a difference between the input current 
frame and the predictive image, and outputting a coded 
picture signal that includes a signal obtained by coding 
said prediction en^or, wherein in the first step, a refer- 
ence frame which satisfies the following condition is se- 
lected as the first reference frame from among the cod- 
ed frames: a reference frame which is located before 
the current frame in display order and which is assigned 
a smallest first relative index, and in the second step, 
( i ) from a first motion vector used for motion-compen- 
sating a first block which is in the second reference 
frame and co-located with a predetermined block In the 
current frame and from an interval between the first ref- 
erence frame and the second reference frame, a second 
motion vector extending from the first block to a block 
which is in the first reference frame and co-located with 
a block refen^ed to by the first motion vector is calculat- 
ed, ( ii ) based on said calculated second motion vector, 
a motion vector to the first reference frame and a motion 
vector to the second reference frame are calculated us- 
ing a predetermined method, and ( iii ) the predictive im- 
age is generated through pixel interpolation from a block 
obtained from the motion vector to the first reference 
frame and a block obtained from the motion vector to 
the second reference frame, 

[0133] In addition to the above, the picture decoding 
method according to the present invention may be a pic- 
ture decoding method, comprising: a first step of obtain- 
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ing a coded picture signal that includes a signal obtained 
by coding a predk:tion error; a second step of selecting 
either at least a first or second reference frame from 
among a plurality of decoded frames stored in a storage 
5 unit by use of a first relative index and a second relative 
index assigned to said decoded frames, in order to se- 
lect a first reference frame and a second reference 
frame to be referred to when obtaining a block on a cur- 
rent frame to be decoded by means of motion compen- 
10 sation; a third step of generating, through pixel interpo- 
lation, a predictive image from a block on at least either 
the first or second reference frame obtained by means 
of motion compensation; a fourth step of generating a 
decoded picture of a frame from the predictive image 
f5 and the decoded prediction error; and a fifth step of stor- 
ing, in the storage unit, a decoded picture of a frame 
with a possibility to be used for inter-frame prediction, 
wherein in the second step, a reference frame which sat- 
isfies the following condition is selected as the first ref- 
erence frame from among the decoded frames: a refer- 
ence frame which is located before the current frame In 
display order and which is assigned a smallest first rel- 
ative index; and in the third step, ( i ) from a first motion 
vector used for motion-compensating a first block which 
is in the second reference frame and co-located with a 
predetermined block in the cun^ent frame and from a 
temporal interval between the first reference frame and 
the second reference frame, a second motion vector ex- 
tending from the first block to a block which is in the first 
reference frame and co-located with a block referred to 
by the first motion vector is calculated, ( ii ) based on 
said calculated second motion vector, a motion vector 
to the first reference frame and a motion vector to the 
second reference frame are calculated using a prede- 
termined method, and ( iii ) the predictive image is gen- 
erated through pixel interpolation from a block obtained 
from the motion vector to the first reference frame and 
a block obtained from the motion vector to the second 
reference frame. 

[0134] In addition to the above, the picture coding 
method according to the present invention may be a pic- 
ture coding method, comprising: a first step of selecting 
either at feast a first or second reference frame from 
among a plurality of coded frames stored in a storage 
unit by use of a first relative index and a second relative 
index assigned to said coded frames, in order to select 
a first reference frame and a second reference frame to 
be referred to when obtaining a block on a cun'ent frame 
to be coded by means of motion compensation; a sec- 
ond step of generating, through pixel interpolation, a 
predictive image from a block on at least either the first 
or second reference frame obtained by means of motion 
compensation; and a third step of coding a prediction 
error which is a' difference between the input current 
freune and the predictive image, and outputting a coded 
picture signal that includes a signal obtained by coding 
said prediction error, wherein in the first step, the first 
reference frame is selected from frames that are located 
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before the current frame in display order, In the second 
step. ( i ) from a first motion vector used for motion-com- 
pensating a first block which is in the second reference 
frame and co-located with a predetennined blocl< in the 
current frame and from a temporal interval between the 5 
first reference frame selected in the first step and the 
second reference frame, a second motion vector ex- 
tending from the first block to a block which is in the first 
reference frame and co-located with a block referred to 
by the first motion vector is calculated, ( li ) based on io 
said calculated second motion vector, a motion vector 
to the first reference frame selected in the first step and 
a motion vector to the second reference frame are cal- 
culated using a predetennined method, and ( iii ) the 
predictive image Is generated through pixel interpolation is 
from a block obtained from the motion vector to the first 
reference frame selected in the first step and a block 
obtained from the motion vector to the second reference 
frame, and i n the third step, the first relative index indi- 
cating the first reference frame selected in the first step 20 
is incorporated into the coded picture signal. 
[0135] In addition to the above, the picture decoding 
method according to the present invention may be a pic- 
ture decoding method, comprising: a first step of obtain- 
ing a 'coded picture signal that includes a signal ob- 25 
tained by coding a prediction error; a second step of se- 
lecting either at least a first or second reference frame 
from among a plurality of decoded frames stored in a 
storage unit by use of a first relative index and a second 
relative index assigned to said decoded frames, in order 30 
to select a first reference frame and a second reference 
frame to be referred to when obtaining a block on a cur- 
rent frame to be decoded by means of motion compen- 
sation; a third step of generating, through pixel interpo- 
lation, a predictive image from a block on at least either 35 
the first or second reference frame obtained by means 
of motion compensation; a fourth step of generating a 
decoded picture of a frame from the predictive image 
and the decoded prediction error; and a fifth step of stor- 
ing, In the storage unit, a decoded picture of a frame 40 
with a possibility to be used for inter-frame prediction, 
wherein in the first step, the first relative index is ob- 
tained from the coded picture signal including first rela- 
tive indices, and in the second step, ( i ) from a first mo- 
tion vector used for motion-compensating a first block 
which is in the second reference frame and co-located 
with a predetemnlned block in the cu n-ent frame and from 
a temporal interval between the first reference frame se- 
lected In the first step and the second reference frame, 
a second motion vector extending from the first block to so 
a block which is in the first reference frame and co-lo- 
cated with a block referred to by the first motion vector 
is calculated, ( ii ) based on said calculated second mo- 
tion vector, a motion vector to the first reference frame 
selected in the first step and a motion vector to the sec- ss 
ond reference frame are calculated using a predeter- 
mined method, and ( ill ) the predictive image Is gener- 
ated through pixel interpolation from a block obtained 



from the motion vector to the first reference frame ob- 
tained in the first step and a block obtained from the mo- 
tion vector to the second reference frame. 
[01 36] In addition to the above, the picture coding ap- 
paratus according to the present invention may be a pic- 
ture coding apparatus, comprising: a difference calcu- 
lation unit operable to obtain a picture signal, calculate 
a difference between said picture signal and a predictive 
image, and output said difference as a residual signal; 
an Image coding unit operable to perform Image coding 
on said differential signal, and output said coded signal 
as a coded residual signal; an image decoding unit op- 
erable to decode said coded residual signal, and output 
said decoded signal as a decoded residual signal; an 
addition unit operable to add the decoded residual sig- 
nal and the predictive image, and output a reconstructed 
image; a direct mode second reference frame selection 
unit operable to select, as a second reference frame, a 
reference frame that satisfies the following condition 
from among coded frames stored in a storage unit: a 
reference frame that is located after a current frame to 
be coded and that is assigned a smallest second relative 
index; a direct mode vector/relative index generation 
unit operable to generate motion vectors to a first refer- 
ence frame and to the second reference frame using a 
predetermined method, based on a motion vector used 
for motion-compensating a block that is in the second 
reference frame and co-located with a predetermined 
block in the current frame, said first reference frame be- 
ing a frame referred to by the motion vector used for said 
motion compensation; a pixel interpolation unit operable 
to perform pixel interpolation on two reference blocks 
referred to by the motion vector to the first reference 
frame and the motion vector to the second reference 
frame, and output the result as the predictive image; and 
a variable length coding unit operable to perform varia- 
ble length coding on a prediction error, and output a cod- 
ed signal. 

[0137] In addition to the above, the picture decoding 
apparatus according to the present Invention may be a 
picture decoding apparatus, comprising: a variable 
length decoding unit operable to obtain a coded picture 
signal, perform variable length decoding on said signal, 
and output coded residual signals; an image decoding 
unit operable to decode each of said coded residual sig- 
nals, and output decoded residual signals; an addition 
unit operable to add each of the decoded residual sig- 
nals and each of predictive image signals, and output 
decoded pictures; a storage unit operable to store said 
decoded pictures; a direct mode second reference 
frame selection unit operable to select, as a second ref- 
erence frame, a reference frame that is assigned a 
smallest second relative Index; a direct mode vector/rel- 
ative index generation unit operable to generate motion 
vectors to a first reference frame and to the second ref- 
erence frame using a predetermined method, based on 
a motion vector referred to by a block that is in the sec- 
ond reference frame and co-located with a current block 
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to be coded, said first reference frame being a franrie 
refen-ed to by the motion vector referred to by said blocic 
collocated with the current block; and a pixel interpola- 
tion unit operable to perform pixel interpolation on the 
first and second reference frames as well as on two ref- s 
erence blocks referred to by the motion vectors corre- 
sponding to the respective reference frames, and output 
the predictive image signal. 

[01 38] Furthennnore, the recording medium according 
to the present invention is a recording medium in which io 
a program is stored by a computer, the program being 
Intended for performing coding or decoding processing 
on a moving picture described in any one of the afore- 
mentioned picture coding and picture decoding meth- 
ods as well as causing the computer to execute the cod- is 
ing or decoding processing on a moving pbture de- 
scribed in any one of the aforementioned picture coding 
and picture decoding methods. 

[0139] Note that It is possible to embody the present 
invention as the following that use the above-described 20 
picture coding method or picture decoding method: a 
picture coding apparatus, a picture decoding apparatus, 
a program, and a recording medium storing such pro- 
gram. 

25 

Brief Description of Drawings 
[0140] 

FIG. 1 is a conceptual diagram showing B pictures. 30 
FIG. 2 is a diagram explaining interpolative predic- 
tion. 

FIG. 3 is a diagram explaining an example case 
where two reference pictures of an interpolation 
predictive block are located before a cun^ent picture 35 
to be coded In display order that includes the inter- 
polation predictive block. 

FIG. 4 is a diagram explaining an example case 
where two reference pictures of an interpolation 
predictive block are located after a current ptoture 40 
to be coded in display order that includes the inter- 
polation predictive block. 

FIGS. 5A and 5B are diagrams explaining picture 
numbers and reference Indices. 

FIG. 6 is a diagram explaining direct mode em- 45 
ployed by a conventional picture coding apparatus. 
FIG. 7 Is a block diagram showing a configuration 
of a conventional picture coding apparatus. 
FIG. 8 is a conceptual diagram showing a format of 
a bit stream. so 
FIG. 9 is a block diagram showing a configuration 
of a conventional picture decoding apparatus. 
FIG. 1 0 is a block diagram showing a picture coding 
apparatus according to a first embodiment of the 
present invention. 55 
FIG. 11 is a diagram showing a partial structure of 
a bit stream according to the first embodiment. 
FIG. 12 is a flowchart showing an operation of cod- 



ing a picture in direct mode in the first embodiment. 
FIGS. 13A and 138 are diagrams explaining an op- 
eration to be performed when remapping takes 
place In the first embodiment. 
FIG. 14 Is a diagram explaining an example display 
order of pictures In the first embodiment. 
FIG. 1 5 is a block diagram showing a picture decod- 
ing apparatus according to a second embodiment 
of the present invention. 

FiG. 16 is a flowchart showing an operation of de- 
coding a picture in direct mode in the second em- 
bodiment. 

FIG. 1 7 is a block diagram showing a picture coding 
apparatus according to a third embodiment of the 
present Invention. 

FIG. 1 8 is a diagram showing a structure of a header 
of a bit stream in the third embodiment. 
FIG. 1 9 is a flowchart showing an operation of cod- 
ing a picture in direct mode in the third embodiment. 
FIG. 20 Is a block diagram showing a picture decod- 
ing apparatus according to a fourth embodiment of 
the present Invention. 

FiG. 21 is a flowchart showing an operation of de- 
coding a picture In direct mode in the fourth embod- 
iment. 

FIG. 22 Is a block diagram showing a picture coding 
apparatus according to a fifth embodiment of the 
present Invention. 

FIG. 23 is a diagram explaining remapping per- 
formed by a management unit in the fifth embodi- 
ment. 

FiG. 24 Is a flowchart showing an operation per- 
formed in the fifth embodiment. 
FIG. 25 is a diagram showing a configuration of a 
picture coding apparatus according to a variation of 
the fifth embodiment. 

FIG. 26 Is a block diagram showing a picture decod- 
ing apparatus according to a sixth embodiment of 
the present invention. 

FIG. 27 is a block diagram showing a picture coding 
apparatus according to a seventh embodiment of 
the present invention. 

FIG. 28 Is a conceptual diagram showing direct 
mode according to the seventh embodiment. 
FIG. 29 Is a flowchart showing a process of calcu- 
lating motion vectors in the seventh embodiment. 
FIG. 30 is a block diagram showing a picture decod- 
ing apparatus according to an eighth embodiment 
of the present Invention. 

FiG. 31 is a block diagram showing a picture coding 
apparatus according to a ninth embodiment of the 
present Invention. 

FIG. 32 shows a format of a bit stream BS3 accord- 
ing to the ninth embodiment. 
FIG. 33 is a block diagram showing a picture decod- 
ing apparatus according to a tenth embodiment of 
the present invention. 

FIGS. 34A, 348, and 34C are diagrams explaining 
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a recording medium according to an eieventh em- 
bodiment of the present invention. 
FIG. 35 is a block diagram showing an overall con- 
figuration of a content supply system that realizes 
a content distribution service according to a twelfth 
embodiment of the present invention. 
FIG. 36 is a diagram showing a cellular phone ac- 
cording to the twelfth embodiment. 
FIG. 37 is a block diagram showing the cellular 
phone according to the twelfth embodiment. 
FIG. 38 is a block diagram showing an overall con- 
figuration of a digital broadcasting system accord- 
ing to the twelfth embodiment 

Best Mode f6r Carrying Out the Invention 

(First Embodiment) 

[01 41 J The following describes the picture coding ap- 
paratus according to the first embodiment of the present 
invention with reference to the figures. 
[0142] FIG. 1 0 is a block diagram showing the picture 
coding apparatus according to the first embodiment. 
[01 43] Such picture coding apparatus 1 00, which ob- 
tains a picture signal Img with the contents representing 
pictures and codes such picture signal Img on a block- 
by-block basis, is comprised of a subtracter 1 01 , an im- 
age decoding unit 104, a variable length coding unit 1 03, 
a motion estimation unit 1 09, a prediction type selection 
unit 108, switches 111 and 112, a pixel Interpolation unit 
106, a direct mode processing unit 110, a vector buffer 
114, a multi frame buffer 107, and a picture selection 
unit 113. 

[01 44] The subtracter 101 subtracts predictive image 
signals Pre from the picture signal Img, and outputs the 
resultant as residual signals Res. 
[01 45] The Image coding unit 1 02 obtains the residual 
signals Res from the subtracter 1 01 so as to perform 
image coding processing such as OCT conversion and 
quantization on each of such residual signals Res, and 
outputs coded residual signals ER that include quan- 
tized DOT coefficients and the like. 
[0146] The Image decoding unit 1 04 obtains the cod- 
ed residual signals ER from the Image coding unit 102 
so as to perform Image decoding processing such as 
inverse quantization and inverse DCT conversion on 
each of such coded residual signals ER, and outputs 
decoded residual signals DR. 

[0147] The adder 105 adds each of the decoded re- 
sidual signals DR and each of the predictive image sig- 
nals Pre, and outputs the resultant as reconstructed im- 
age signals Rec. 

[0148] Upon the receipt of the reconstructed image 
signals Rec from the adder 105. the multi frame buffer 
107 stores signals with the possibility to be refen^ed to 
for inter-picture prediction of subsequent pictures, out 
of such received reconstructed image signals Rec. 
Since the multi frame buffer 107 has a limited amount 
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of memory, the multi frame buffer 107 removes picture 
data whteh will not be used for later inter-picture predic- 
tion, out of the reconstructed image signals Rec ob- 
tained from the adder 105. 

5 [0149] The motion estimation unit 109 obtains a pic- 
ture stored In the multi frame buffer 907 as a reference 
picture RP so as to perform motion estimation on it, se- 
lects an optimum prediction type for such picture using 
a predetennined method (a prediction type to be select- 

10 ed depends on a picture type) from among the following 
prediction types: intra picture prediction, first reference 
picture prediction, second reference picture prediction, 
and interpolatlve prediction. Here, the motion estimation 
unit 109 selects, for example, a prediction type as the 

IS optimum prediction type that generates the smallest 
prediction error among all the prediction types. 
[0150] When interpolatlve prediction is selected, the 
motion estimation unit 109 outputs the first motion vec- 
tor MV1 , the second motion vector MV2, the first refer- 

20 ence Index RI1 , and the second reference Index RI2 of 
the current block to be coded. When Intra picture pre- 
diction is selected, the motion estimation unit 109 out- 
puts no motion vectors and reference indices as de- 
scribed above. When first reference picture prediction 

25 is selected, the motion estimation unit 1 09 outputs only 
the first motion vector MV1 and the first reference index 
RI1 . When second reference picture prediction Is select- 
ed, the motion estimation unit 109 outputs only the sec- 
ond motion vector MV2 and the second reference index 

30 RI2. 

[0151] The vector buffer 114 obtains, from the motion 
estimation unit 1 09, the first motion vector MV1 , the sec- 
ond motion vector MV2, the first reference index RI1 , 
and the second reference index RI2 on a block-by-block 

35 basis. Then, on the basis of such obtalnment, the vector 
buffer 114 stores scaling vectors rMV used for direct 
mode as well as the picture numbers and reference in- 
dices indicating pictures referred to by the respective 
scaling vectors rMV. 

40 [0152] In other words, In direct mode, since a first mo- 
tion vector used to code a block In a coded picture is 
used as a scaling vector rMV and a picture referred to 
by such first motion vector serves as Its first reference 
picture, the vector buffer 1 14 stores first motion vectors 

45 MV1 and first reference Indices RM with the possibility 
to be used for direct mode, out of the first motion vectors 
MV1 , the second motion vectors M V2, the first reference 
indices RI1 , and the second reference indices RI2 out- 
putted from the motion estimation unit 109. 

so [0153] Furthermore, the vector buffer 114 obtains a 
second reference index rRI2 from the picture selection 
unit 1 13. After obtaining the second reference Index rRI2 
from the picture selection unit 1 1 3, the vector buffer 1 1 4 
outputs, as a scaling vector rMV, the first motion vector 

S5 MV1 of a predetennined block Included in a picture In- 
dicated by such second reference index rlR2, and out- 
puts the first reference Index RM Indicating a picture re- 
ferred to by such scaling vector riVIV as a first reference 
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Index rRII. 

[0154] After obtaining a direct mode scaling coeffi- 
cient SP, a scaling vector rMV, a first reference index 
rRI1, and a second reference Index rRI2, the direct 
mode processing unit 1 1 0 perfomns processing based 
on the above-described direct mode on the cun-ent block 
to be coded, and outputs the resultant as a first motion 
vector sMVI , a second motion vector sM V2, a first ref- 
erence index rRII , and a second reference index rRI2. 
Here, Instead of obtaining the direct mode scaling coef- 
ficient SP, the direct mode processing unit 1 1 0 may ob- 
tain display time Information that indicates the display 
time of each of the reference pictures specified by the 
first reference indices rRI1 and display time information 
that indicates the display time of each of the reference 
pictures specified by the second reference indices rRI2 
as well as display time Information indicating the display 
time of the current picture to be coded. In this case, the 
direct mode processing unit 110 shall calculate the dif- 
ference in display times between each picture from the 
display time of each picture Indicated by such display 
time information, and performs scaling on the scaling 
vector rMV using the ratio of the display time difference 
between each of the pictures. Accordingly, the direct 
mode processing unit 110 can detemiine a first motion 
vector sMVI and a second motion vector sMVs as In the 
case of using a direct mode scaling coefficient SP. 
[0155] The prediction type selection unit 108 obtains 
( 1 ) the picture signal Img, ( ii ) the reference pictures 
RP, and ( Hi ) the first reference indices rRII , the second 
reference indices rRI2. the first motion vectors sMVI , 
and the second motion vectors sM V2 outputted from the 
direct mode processing unit 110, as well as obtaining 
the first reference indices RI1 , the second reference In- 
dices RI2, the first motion vectors MV1 , and the second 
motion vectors MV2 outputted from the motion estima- 
tion unit 1 09. Then, the prediction type selection unit 1 08 
selects a prediction type for the current block to be cod- 
ed based on such obtained information. In other words, 
the prediction type selection unit 1 08 Judges whether to 
apply direct mode or not to such current block to be cod- 
ed. Furthermore, the prediction type selection unit 1 08 
outputs, to the variable length coding unit 108, type In- 
formation PT indicating the selected prediction type. 
[0156] Here, in order to select a selection type, the 
prediction type selection unit 108 compares, for exam- 
ple, prediction errors at the time of applying/not applying 
direct mode to the current block. When judging that a 
prediction error is smaller when direct mode is applied, 
the prediction type selection unit 108 selects a predic- 
tion type that utilizes direct mode. When judging, on the 
other hand, that a prediction error is smaller when direct 
mode is not applied, the prediction type selection unit 
1 08 selects a prediction type that does not utilize direct 
mode. 

[0157] Furthenmore, when selecting to apply direct 
mode as a prediction type, the prediction type selection 
unit 108 causes the switch 111 to change the contact 



from 0 to 1. Accordingly, a tirst reference index rRII . a 
second reference index rRI2, a first motion vector sMVI , 
and a second motion vector sMV2 outputted from the 
direct mode processing unit 110 are transmitted to the 

5 multi frame buffer 1 07 via the switch 111 as a first refer- 
ence index RI1, a second reference index RI2, a first 
motion vector MV1 , and a second motion vector MV2. 
[0158] Meanwhile, when selecting not to apply direct 
mode as a prediction type, the prediction type selection 

10 unit 108 causes the switch 111 to change the contact 
from 1 to 0. Accordingly, a first reference index RI1, a 
second reference index RI2, a first motion vector MVI , 
and a second motion vector MV2 outputted from the mo- 
tion estimation unit 109 are transmitted to the variable 

15 length coding unit 103 and the multi frame buffer 107 
via the switch 111. 

[0159] Furthermore, when selecting a prediction type 
other than interpolatlve prediction, the prediction type 
selection unit 1 08 causes the switch 112 to change the 

20 contact from 1 to 0, whereas the prediction type selec- 
tion unit 108 causes the switch 112 to change the con- 
tact from 0 to 1 when selecting interpolative prediction 
as a prediction type. As a result, predictive image sig- 
nals Pre obtained by the subtracter 101 and the adder 

25 105 are assorted into ones outputted from the multi 
frame buffer 107 and ones from the pixel interpolation 
unit 106, depending on the prediction type of each pre- 
dictive image signal Pre. 

[01 60] At the time of interpolative prediction, the multi 
30 frame buffer 107 obtains a first reference index RI1 
(rRII) and a first motion vector MVI (sMVI) based on 
the outputs from the motion estimation unit 1 09 and the 
direct mode processing unit 1 1 0, and outputs, to the pix- 
el interpolation unit 1 06, a reference block RBI corre- 
35 spending to such first reference index RI1 and first mo- 
tion vector MVI . Furthennore, the multi frame buffer 107 
obtains a second reference index RI2 (rR12) and a sec- 
ond motion vector MV2 (sMV2) based on the outputs 
from the motion estimation unit 1 09 and the direct mode 
40 processing unit 1 1 0, and outputs, to the pixel Interpola- 
tion unit 106. a reference block RB2 corresponding to 
such second reference index RI2 and second motion 
vector MV2. 

[0161] At the time of first reference picture prediction, 
the multi frame buffer 1 07 outputs a reference block RB 
corresponding to the first reference index RI1 and the 
first motion vector MVI . Meanwhile, at the time of sec- 
ond reference picture prediction, the multi frame buffer 
107 outputs a reference block RB corresponding to the 

so second reference index RI2 and the second motion vec- 
tor MV2. Note that when intra picture prediction is se- 
lected, the multi frame buffer 107 outputs a block RB 
that is made up of pixels generated as a result of intra 
picture prediction. 

ss [0162] After obtaining the reference blocks RBI and 
RB2 from the multi frame buffer 107, the pixel interpo- 
lation unit 1 06 interpolates the pixel values of co-located 
pixels of the respective reference blocks RBI and RB2, 
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and outputs the resultant as an inteipolated block ReP. 
[01 63] The contacts of the switch 11 2 are changed ac- 
cording to a prediction type indicated by the prediction 
type selection unit 108 as described above. Therefore, 
when prediction other than interpolative prediction is se- 
lected as a prediction type, a block RB from the multi 
frame buffer 1 07 is transmitted to the subtracter 1 01 and 
the adder 105 via the switch 112 as a predictive image 
signal Pre. When interpolative prediction Is selected as 
a prediction type, on the other hand, an Interpolated 
block ReP from the pixel interpolation unit 106 is trans- 
mitted to the subtracter 101 and the adder 1 05 via the 
switch 112 as a predictive image signal Pre. 
[0164] The variable length coding unit 103 performs 
variable length coding on the coded residual signals ER, 
the first reference indices RI1 , the second reference In- 
dices Rt2, the first motion vectors MV1 , the second mo- 
tion vectors MV2, the direct mode seating coefficients 
SP, and the type information PT, and outputs the result- 
ant as a bit stream BS. In the case where display time 
information is used by the direct mode processing unit 
110 instead of the direct mode seating coefficients SP, 
the variable length coding unit 103 performs variable 
length coding on the display time Infonnatlon used for 
the above-perfomied scaling without variable length 
coding the direct mode scaling coefficients SP, and In- 
corporates the variable length-coded display time infor- 
mation Into the bit stream BS. 

[0165] FIG. 11 Is a diagram showing a partial structure 
of the bit stream BS. 

[0166] The bit stream BS is structured on a picture- 
by-picture basis, where Information about the header 
and blocks is included on a picture-by-plcture basis. 
[01 67] The part of the bit stream BS shown in FIG. 1 1 , 
which is the equivalent of a picture that Includes blocks 
coded in direct mode, is made up of a header 21 , a cod- 
ed first block signal 22, and a coded second block signal 
23. 

[01 68] The coded first block signal 22, which indicates 
a block coded in direct mode, Includes a coded type sig- 
nal 28 obtained by coding type infomnation PT that Indi- 
cates the use of direct mode. 

[01 69] The coded second block signal 23, which indi- 
cates a block coded In interpolative prediction other than 
in direct mode, includes: a coded type signal 29 ob- 
tained by coding type infonnatlon PT that indicates the 
non-use of direct mode; a coded first Index signal 30 
obtained by coding a first reference index RI1 ; a coded 
second index signal 31 obtained by coding a second ref- 
erence index RI2; a coded MV1 signal 32 obtained by 
coding a first motion vector IV^VI ; and a coded MV2 sig- 
nal 33 obtained by coding a second motion vector MV2. 
[0170] The header 21 includes: coded remapping in- 
formation 27 obtained by coding remapping information 
indk:ating a change in the assignment of reference indi- 
ces to the picture; a coefficient TR1 included In a direct 
mode scaling coefficient SP shown in FIG. 6, i.e. first 
time difference Infonnatlon 24 obtained by coding the 



difference in display times between the current picture 
to be coded and the first reference picture; a coefficient 
TR2 included in the direct mode scaling coefficient SP, 
i.e. second time difference infonmation 25 obtained by 
5 coding the difference in display times between the cur- 
rent picture to be coded and the second reference pic- 
ture; and a coefficient TRO included in the direct mode 
scaling coefficient SP, I.e. time difference information 26 
obtained by coding the difference in display times be- 
10 tween the first reference picture and the second refer- 
ence picture. Note that when the direct mode processing 
unit 1 1 0 uses not the direct mode scaling coefficient SP 
but display time information, the header includes a sig- 
nal obtained by coding the display time information of 
'5 the picture, instead of the first time difference informa- 
tion 24, the second time difference information 25 and 
the time difference information 26. 
[0171] Here, the picture selection unit 113 In the 
present embodiment selects, as a second reference pic- 
ture, a picture to be used for direct mode from among 
pictures stored in the multi frame buffer 107. 
[0172] Stated another way, the picture selection unit 
113 selects, as the second reference picture for direct 
mode, a picture with the smallest second reference In- 
dex among the pictures located after the current picture 
to be coded in display order. 

[0173] Then, the picture selection unit 113 outputs, to 
the vector buffer 114 and the direct mode processing 
unit 110, the second reference index rRI2 of such se- 
lected second reference picture. Note that the picture 
selection unit 113 can know about the display order of 
each picture by using infomnation about display order 
attached to each picture. 

[01 74] FIG . 1 2 is a flowchart showing the operation of 
coding a picture in direct mode in the present embodi- 
ment. 

[0175] First, the picture coding apparatus 1 00 causes 
the picture selection unit 1 1 3 to select a picture with the 
smallest second reference index as the second refer- 
ence picture, from among'the subsequent pictures of the 
current picture to be coded in display order (Step SI 00). 
[01 76] Next, the picture coding apparatus 1 00 causes 
the vector buffer 11 4 to specify, as a scaling vector rMV, 
the first motion vector MV1 of a block that is co-located 
with the cun-ent block and included in the second refer- 
ence picture selected in the above manner, and to spec- 
ify, as a first reference picture, a picture referred to by 
such first motion vector MV1 (Step SI 02). 
[0177] Then, the picture coding apparatus 1 00 gener- 
ates a predictive image in direct mode through the direct 
mode processing unit 110, the multi frame buffer 107, 
and the pixel interpolation unit 1 06. utilizing the second 
reference picture selected in the above manner as well 
as the first reference picture and the scaling vector rMV 
specified in the above manner (Step SI 04). 
[0176] After generating the predictive image, the pic- 
ture coding apparatus 100 generates a prediction error 
from the current block to be coded and such predictive 
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image (Step S106), and codes such generated predic* 
tion error (Step S108). 

[01 79] Here, a description is given of the operation to 
be performed in the first embodiment when a change is 
made in first reference indices and second reference in- 
dices to be assigned to other pictures with respect to the 
current picture to be coded (such operation is hereinaf- 
ter refereed to as "remapping"). 
[0180] FIGS. 13A and 13B are diagrams explaining 
the operation to be carried out In the first embodiment 
when remapping takes piace. 

[0181] FIG. 13A shows first reference indices and 
second reference indices assigned to the respective pic- 
tures before remapping tal^es place (in the default 
state). 

[0182] Before remapping takes place, i.e. in the de- 
fault state, 0, 1 , and 2 are assigned as first reference 
indtees to all the pictures preceding the current picture 
in display order which is indicated by dotted lines in FIG. 
13A, in order of proximity to such current picture to be 
coded. More specifically, 0 is assigned as a first refer- 
ence index to a picture with the picture number 14, 1 is 
assigned as a first reference index to a picture with the 
picture number 12, and 2 is assigned as a first reference 
Index to a picture with the picture number 10. 
[0183] At the completion of assigning first reference 
indices to all the pictures preceding the current picture 
in display order, then, 3, 4, and 5 are assigned as first 
reference indices to all the pictures located after the cur- 
rent picture in display order, in order of proximity to such 
current picture to be coded. More specifically, 3 is as- 
signed as a first reference index to a picture with the 
picture number 15, 4 Is assigned as a first reference In- 
dex to a picture with the picture number 13, and 5 Is 
assigned as a first reference index to a picture with the 
picture number 11. 

[01 84] Subsequently, 0, 1 , and 2 are assigned as sec- 
ond reference indices to all the pictures located after the 
current picture in display order, in order of proximity to 
such current picture to be coded. More specifically, 0 is 
assigned as a second reference index to a picture with 
the picture number 15, 1 Is assigned as a second refer- 
ence index to a picture with the picture number 13, and 
2 is assigned as a second reference index to a picture 
with the picture number 1 1 . 

[0185] At the completion of assigning second refer- 
ence indices to all the pictures located after the current 
picture in display order, then, 3, 4, and 5 are assigned 
as second reference Indices to all the pictures preceding 
the current pteture In display order, in order of proximity 
to such current picture to be coded. More specifically, 3 
is assigned as a second reference Index to a picture with 
the picture number 14, 4 is assigned as a second refer- 
ence index to a picture with the picture number 12, and 
5 Is assigned as a second reference index to a picture 
with the picture number 10. 

[0186] FIG. 138 shows first reference indices and 
second reference Indices assigned to the respective pk;- 



tures as well as remapping Infomiatlon In the case 
where remapping was carried out. 
[0187] When remapping takes place, the first refer- 
ence index assigned to a picture with the picture number 
5 "14" is changed, for example, from "0" to "2", the first 
reference Index assigned to a picture with the picture 
number "12" is changed from "1 " to "0", and the first ref- 
erence index assigned to a picture with the picture 
number "10" is changed from "2" to "1". Moreover, the 

10 second reference index assigned to a picture with the 
picture number "14" Is changed from "3" to "0", the sec- 
ond reference index assigned to a picture with the pic- 
ture number "1 5" is changed from "0" to "2", and the sec- 
ond reference index assigned to a picture with the pic- 
's ture number "11" is changed from "2" to "3". 

[0188] Remapping information is made up of numer- 
ical values indicating changes in reference indices of re* 
spective pictures subject to such changes. Remapping 
infomiation shown in FIG. 138 is for the second refer- 

20 ence indices. For example, when the picture which is 
assigned the second reference index of "0" is changed 
from a picture with the picture number "15" to a picture 
with the picture number "14", the value "-1 " obtained by 
subtracting the picture number "15" from the picture 

25 number "14" shall be included in the remapping infor- 
mation in association with the picture number "14". 
Moreover, when the picture which is assigned the sec- 
ond reference index of "2" is changed from a picture with 
the picture number "11" to a picture with the picture 

30 number "15", the value "+4" obtained by subtracting the 
picture number "11" from the picture number "15" shall 
be included in the remapping infomnation In association 
with the picture number "1 5". Similarly, when the picture 
which is assigned the second reference index of "3" Is 

35 changed from a picture with the picture number "14" to 
a picture with the picture number "11", the value "-3" ob- 
tained by subtracting the picture number "14" from the 
picture number "11" shall be included In the remapping 
Information in association with the picture number "11". 

40 Note that when remapping as shown in FIG. 1 38 is car- 
ried out, the picture which is assigned the second refer- 
ence index of "1 " shall not be changed, but the value "0" 
is included in the remapping information in association 
with the picture number "13". 

^5 [0189] Such remapping information is managed by 
the management unit In the picture coding apparatus 
1 00 not illustrated in the figures, and Is used by the mo- 
tion estimation unit 109, the prediction type selection 
unit 108, the multl frame buffer 1 07, and the pteture se- 

50 lection unit 1 1 3 to specify the first reference Indices and 
second reference indices. 

[0190] In the present embodiment with the above con- 
figuration, the picture selection unit 113 selects, from 
among the pictures located after the current picture in 
55 display order, a picture with the smallest second refer- 
ence index as the second reference picture. Therefore, 
when no remapping takes place, a picture with the pic- 
ture number "15" and with the second reference index 
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"0" is used as a second reference picture in direct mode, 
whereas when remapping does take piace, a picture 
with the picture number "13" and with the second refer- 
ence index "1" is used as a second reference picture in 
direct mode. 5 
[0191] Accordingly, even when remapping takes 
place, since a picture located after the current picture is 
used as a second reference picture at the time of coding 
in direct mode, it is not possible for forward two pictures 
or backward two pictures to be used as first reference io 
pictures or second reference pictures. This allows a for- 
ward picture and a backward picture to be used respec- 
tively as a first reference picture and a second reference 
picture without fail, which consequently leads to an im- 
proved coding efficiency. is 
[01 92] Here, a description is given of the memory ca- 
pacity of the picture coding apparatus 100 according to 
the present embodiment. 

[0193] FIG. 14 is a diagram explaining an example 
display order of pictures. 20 
[0194] As FIG. 14 shows, pictures P1~P7 are dis- 
played in the order of the picture PI , the picture P3, the 
picture P4, the picture P2, the picture P6, the picture P7, 
and the picture P5, and are coded in the order of the 
picture P1 , the picture P2, the picture P3, the picture P4, 25 
the picture P5, the picture P6, and the picture P7 to be 
stored into the multi frame buffer 1 07. 
[01 95] Here, B pictures shall not be used as reference 
pictures. Therefore, the multi frame buffer 1 07 stores the 
pictures PI , P2, and P5 which are either I picture or P 30 
picture to be handled as reference pictures, and deletes 
the pictures P3, P4, P6, 'and P7 which are B pictures 
not to be handled as reference pictures. 
[0196] When the pictures P3 and P4 are coded, the 
reference picture P2 serves as a reference picture which 35 
is located later in display order than such current pic- 
tures to be coded. Meanwhile, when the pictures P6 and 
P7 are coded, the reference picture P2 serves as a ref- 
erence picture which is located earlier in display order 
than such current pictures to be coded. 40 
[0197] When pictures have such a relationship as 
shown in FIG. 1 4, the reference picture P2 cannot serve 
as a reference picture located after a current picture to 
be coded in display order when coding the pictures sub- 
sequent to such reference picture P2. In other words, 45 
once a reference picture comes before a current picture 
to be coded in display order, such reference picture shall 
never come after a current picture to be coded in display 
order again. 

[0198] Stated another way, at the point of time when so 
a picture stored in the multi frame buffer 1 07 comes be- 
fore a current picture in display order, it becomes pos- 
sible to delete such reference picture's first motion vec- 
tors and information used to identify a picture referred 
to by each of such first motion vectors stored in the vec- 
tor buffer 114. 

[01 99] Therefore, when pictures have such a relation- 
ship as shown in FIG. 14. the vector buffer 114 is re- 
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quired to hold only the first motion vectors of the refer- 
ence pictures located after the current picture in display 
order as well as information used to identify pictures re- 
ferred to by such first motion vectors. Accordingly, when 
pictures have a relationship as shown in FIG. 14, it be- 
comes possible to reduce the amount of memory re- 
quired for the vector buffer 1 1 4. 
[0200] Note that in the present embodiment, a picture 
with the smallest second reference Index Is selected as 
a second reference picture in direct mode, from among 
pictures that are located after the current picture to be 
coded in display order. However, a picture to be selected 
as a second reference picture does not have to be a 
picture with the smallest reference Index, as long as 
such picture to be selected satisfies a condition under 
which a second reference picture in direct mode can be 
uniquely detemiined from among reference pictures lo- 
cated after the current picture in display order. In such 
case too, there arises no possibility to refer to only pic- 
tures preceding the current picture to be coded, which 
results in an increased coding efficiency. Furthemiore, 
a second reference picture in direct mode may also be 
a picture which is closest to the current picture to be 
coded In display order among the reference pictures 
subsequent to such current picture in display order. This 
can also improve coding efficiency as in the above case. 
[0201] Furthermore, it is also possible that a second 
reference picture for direct mode is a picture with the 
smallest second reference index among the pictures 
that are located after the current picture to be coded in 
display order and further that a first reference picture for 
direct mode is a picture with the smallest first reference 
index among the pictures that are located before the cur- 
rent picture to be coded in display order Coding efficien- 
cy can be improved in this case too, as in the above 
case. 

(Second Embodiment) 

[0202] The following describes the picture decoding 
apparatus according to the second embodiment of the 
present invention with reference to the figures. 
[0203] FIG, 15 is a block diagram showing the picture 
decoding apparatus according to the second embodi- 
ment. 

[0204] Such picture decoding apparatus 150, which 
decodes the bit stream BS outputted from the picture 
coding apparatus 100 In the first embodiment, is com- 
prised of a variable length decoding unit 151 , an image 
decoding unit 152, an adder 153, switches 155 and 156, 
a direct mode processing unit 154, a pixel interpolation 
unit 157, a multi frame buffer 158, a vector buffer 160, 
and a picture selection unit 159. 
[0205] The variable length decoding unit 151 obtains 
the bit stream BS from outside the picture decoding ap- 
paratus 150, performs variable length decoding on such 
bit stream BS, and outputs the coded residual signals 
ER, the motion vectors MV1 and MV2, the reference in- 
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dices RI1 and RI2, the direct mode scaiing coefficients 
SP, and the type infomnation PT. Here, when the bit 
stream BS does not include coded direct mode seating 
coefficients SP but coded display time infomriation, the 
variable length decoding unit 151 outputs the display 
time infonnation instead of the direct mode scaling co- 
efficients SP. 

[0206] Furthemiore, the variable length decoding unit 
151 changes the contacts of the switches 155 and 156 
according to a prediction type Indicated by each type 
infomnation PT. In other words, when a prediction type 
is direct mode, the variable length decoding unit 151 
changes the contact of the switch 155 from "0" to "1", 
whereas when a prediction type is other than direct 
mode, the variable length decoding unit 151 changes 
the contact of the switch 1 55 from "1 " to "0". Meanwhile, 
when a prediction type is interpolative prediction, the 
variable length decoding unit 151 changes the contact 
of the switch 156 from "1" to "0", whereas when a pre- 
diction type is other than interpolative prediction, the 
variable length decoding unit 151 changes the contact 
of the switch 156 from "0" to "1 

[0207] The image decoding unit 1 52 obtains the cod- 
ed residual signals ER from the variable length decoding 
unit 151 so as to perform image decoding processing 
such as inverse quantization and inverse DCT conver- 
sion on each of such coded residual signals ER, and 
outputs decoded residual signals DR. 
[0208] The adder 153 adds each of the decoded re- 
sidual signals DR and each of the predictive Image sig- 
nals Pre, and outputs the resultant as decoded image 
signals Dim to outside the picture decoding apparatus 
150. 

[0209] Upon the receipt of the decoded image signals 
Dim, the multl frame buffer 1 58 stores signals with the 
possibility to be referred to for inter-picture prediction of 
subsequent pictures, out of the decoded image signals 
Dim. Since the multi frame buffer 158 has a limited 
amount of memory, the multi frame buffer 158 removes 
picture data which will not be used for later inter-picture 
prediction, out of the above-obtained decoded Image 
signals Dim. 

[0210] The vector buffer 1 60 obtains, from the motion 
estimation unit 109, the first motion vectors IVIVI, the 
second motion vectors IVIV2, the first reference indices 
RI1, and the second reference indices RI2. Then, the 
vector buffer 1 60 stores scaling vectors used for direct 
mode as well as the picture numbers and reference in- 
dices Indicating pictures referred to by the respective 
scaling vectors. 

[0211] In other words, in direct mode, since a first mo- 
tion vector f^VI used to decode a block in a decoded 
picture is used as a scaling vector and a picture referred 
to by such first motion vector IVIVI serves as its first ref- 
erence picture, the vector buffer 160 stores first motion 
vectors MV1 and first reference indices RI1 with the pos- 
sibility to be used for direct mode, out of the first motion 
vectors MV1 , the second motion vectors M V2, the first 



reference indices RI1 , and the second reference Indices 
RI2 outputted from the variable length decoding unit 
151. 

[0212] Furthermore, the vector buffer 160 obtains a 
5 second reference index rRI2 from the picture selection 
unit 159. After obtaining the second reference index 
rRI2 from the picture selection unit 1 59, the vector buffer 
160 outputs, as a scaling vector rMV, the first motion 
vector MV1 of a predetemilned block included in a pic- 
10 ture indicated by such second reference Index rlR2, and 
outputs the first reference index RI1 indicating a picture 
referred to by such scaling vector rMV as a first refer- 
ence index rRI1 . 

[0213] After obtaining a direct mode scaling coeffi- 

15 clent SP, a scaling vector rMV, and a first reference index 
rRII, the direct mode processing unit 154 performs 
processing based on the above-described direct mode 
on the current block to be decoded, and outputs, as the 
resultant, a first motion vector sMVI , a second motion 

20 vector sMV2, a first reference Index rRH , and a second 
reference index rRI2. Here, when obtaining display time 
information of each picture instead of the direct mode 
scaiing coefficient SP, the direct mode processing unit 
154 shall calculate the difference In display times be- 

25 tween each picture from the display time of each picture 
indicated by such display time information, performs 
scaling on the scaling vector rMV using the ratio of the 
display time difference between each of the pictures. 
Accordingly, the direct mode processing unit 154 can 

30 detemriine a first motion vector sMVI and a second mo- 
tion vector sMV2 as in the case of using a direct mode 
scaling coefficient SP. 

[0214] At the time of Interpolative prediction, the multi 
frame buffer 158 obtains a first reference index RI1 

35 (rRll) and a first motion vector MV1 (sMV1) based on 
the outputs from the variable length decoding unit 151 
and the direct mode processing unit 154, and outputs, 
to the pixel interpolation unit 1 57, a reference block RBI 
con'esponding to such first reference index RI1 and first 

40 motion vector MV1 . Furthemnore, the multi frame buffer 
158 obtains a second reference index RI2 (rRI2) and a 
second motion vector M V2 (sM V2) based on the outputs 
from the variable length decoding unit 1 51 and the direct 
mode processing unit 154, and outputs, to the pixel in- 

45 terpolation unit 1 57, a reference block RB2 correspond- 
ing to such second reference index RI2 and second mo- 
tion vector MV2. 

[021 5] At the time of first reference picture prediction, 
the multi frame buffer 1 58 outputs a reference block RB 

50 corresponding to the first reference index RI1 and the 
first motion vector MV1 . Meanwhile, at the time of sec- 
ond reference picture prediction, the multi frame buffer 
158 outputs a reference block RB corresponding to the 
second reference index RI2 and the second motion vec- 

55 tor MV2. Note that when intra picture prediction Is se- 
lected, the multi frame buffer 158 outputs a block RB 
that is made up of pixels generated as a result of intra 
picture prediction. 
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[0216] After obtaining the reference blocks RBI and 
RB2 from the multl frame buffer 158, the pixel interpo- 
lation unit 157 generates interpolated pixeis by deter- 
mining the average value of the pixel values of co-locat- 
ed pixels of the respective reference bloclcs RB1 and 
RB2, and outputs the resultant as an interpolated block 
ReP. 

[021 7] As described above, the contacts of the switch 

1 56 are changed according to a prediction type. There- 
fore, when a prediction type other than interpolative pre- 
diction is selected, a block RB from the multi frame buff- 
er 1 58 is transmitted to the adder 1 55 via the switch 1 56 
as a predictive image signal Pre, whereas when inter- 
polative prediction is selected as a prediction type, an 
Interpolated block ReP from the pixel interpolation unit 

157 is transmitted to the adder 153 via the switch 156 
as a predictive image signal Pre. 

[0218] Here, the picture selection unit 159 in the 
present embodiment selects, from among pictures 
stored In the multi frame buffer 1 58, a picture to be used 
for direct mode as the second reference picture. 
[0219] More specifically, the picture selection unit 1 59 
selects a picture with the smallest second reference in- 
dex as the second reference picture for direct mode, 
from among the subsequent pictures of the current pk:- 
ture to be decoded in display order. 
[0220] Then, the picture selection unit 159 outputs, to 
the vector buffer 160, the second reference index rRI2 
of such selected second reference picture. Note that the 
picture selection unit 159 can know about the display 
order of each picture by using information about display 
order attached to each picture. 

[0221] Accordingly, the picture decoding apparatus 
150 according to the present embodiment can correctly 
decode the bit stream BS outputted by the picture cod- 
ing apparatus ICQ according to the first embodiment. 
More specifically, it is possible for the picture decoding 
apparatus 1 50 to decode the bit stream outputted by the 
picture coding apparatus that selects, from among pic- 
tures located after the cun^ent picture to be coded In dis- 
play order, a picture with the smallest second reference 
index as the second reference picture to be used for di- 
rect mode. 

[0222] FIG. 1 6 is a flowchart showing the operation of 
decoding a picture in direct mode in the present embod- 
iment. 

[0223] First, the picture decoding apparatus 150 
causes the picture selection unit 159 to select a picture 
with the smallest second reference index value as the 
second reference picture, from among the subsequent 
pictures of the cunrent picture to be decoded in display 
order (Step SI 50). 

[0224] Next, the picture decoding apparatus 150 
causes the vector buffer 1 60 to specify, as a scaling vec- 
tor rMV, the first motion vector of a block that Is co-lo- 
cated with the current block to be decoded and included 
In the second reference picture selected in the above 
manner, and to specify, as a first reference picture, a 



picture referred to by such first motion vector MV1 (Step 
SI 52). 

[0225] Then, the picture decoding apparatus 1 50 gen- 
erates a predictive image in direct mode through the di- 
5 rect mode processing unit 154, the multi frame buffer 
158, and the pixel interpolation unit 157, utilizing the 
second reference picture selected in the above manner 
as well as the first reference picture and the scaling vec- 
tor rMV specified in the above manner (Step SI 54). 
10 [0226] Then, the picture decoding apparatus 150 
adds such predictive image to residual data generated 
by decoding the bit stream BS (Step S156). 
[0227] As described above, in the present embodi- 
ment, since a picture with the smallest reference index 
IS Is selected as a second reference picture from among 
the pictures located after the current picture to be de- 
coded in display order, it is possible to correctly decode 
the bit stream which has been coded by the picture cod- 
ing apparatus according to the present invention de- 
scribed in the first embodiment. Moreover, it is possible 
to obtain the same effect as is described in the first em- 
bodiment. 

[0226] Note that the bit stream BS includes the coded 
remapping infomiation 27 as described in the first em- 
bodiment, and the multi frame buffer 158 specifies pic- 
tures indicated by the first reference indices and second 
reference indices, based on remapping Information ob- 
tained by decoding such coded remapping information 
27. 

(Third Embodiment) 

[0229] The following describes the picture coding ap- 
paratus according to the third embodiment of the 
present invention with reference to the figures. 
[0230] FIG. 1 7 is a block diagram showing the picture 
coding apparatus according to the third embodiment. 
[0231] Such picture coding apparatus 200, which ob- 
tains a picture signal Img with the contents representing 
pictures and codes such picture signal Img on a block- 
by-block basis, is comprised of the subtracter 101 , the 
image decoding unit 104, a variable length coding unit 
203, the motion estimation unit 109, the prediction type 
selection unit 108, the switches 111 and 112, the pixel 
interpolation unit 106, the direct mode processing unit 
110, the vector buffer 114, the multi frame buffer 107, 
and a picture selection unit 213. 
[0232] IHere, the subtracter 1 01 , the image decoding 
unit 104, the motion estimation unit 109, the prediction 
type selection unit 108, the switches 111 and 112, the 
pixel interpolation unit 106, the direct mode processing 
unit 1 1 0, the multi frame buffer 1 07, and the vector buffer 
114 in the picture coding apparatus 200 respectively 
have the same functions as those of the picture coding 
apparatus 1 00 according to the first embodiment. 
[0233] Stated another way, the present embodiment 
is characterized by the functions of the picture selection 
unit 213 and the variable length coding unit 203. 
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[0234] The picture selection unit 21 3 selects a picture 
located after the current picture to be coded in display 
order as a second reference picture used for direct 
mode, from among pictures stored in the multi frame 
buffer 107. 

[0235] Then, the picture selection unit21 3 outputs the 
second reference index rRI2 indicating such selected 
second reference picture to the vector buffer 114, the 
direct mode processing unit 1 1 0, and the variable length 
coding unit 203. 

[0236] Note that the picture selection unit 213 can 
know about the display order of each picture by using 
Infomnation about display order attached to each pic- 
ture. Furthemriore, the picture selection unit 213 selects 
one second reference picture used In direct mode for 
the current picture to be coded. In otherwords, the same 
second reference picture is selected for all the blocks 
included in the current picture to be coded to which di- 
rect mode shall be applied. 

[0237] Here» when remapping takes place in the 
present embodiment, the picture selection unit 213 
specifies the first reference Index and the second refer- 
ence index assigned to each picture in the multi frame 
buffer 107 based on remapping infomnation, as in the 
case of the first embodiment. 

[0238] For example, when remapping Is not can-led 
out, as shown In FIG. 13 A, the picture selection unit 21 3 
selects, as the second reference picture for the current 
picture to be coded In direct mode, any one of the sub- 
sequent pictures of the current block to be coded, i.e. 
any one of the picture with the picture number "15", the 
picture with the picture number " 1 3", and the picture with 
the picture number "11". Then, the picture selection unit 
213 outputs the second reference index rRI2 indicating 
such selected picture. For example, when selecting the 
picture with the picture number "15" as the second ref- 
erence picture, the picture selection unit 21 3 outputs the 
second reference index rRI2 indicating 0. 
[0239] Meanwhile, when remapping is candied out, as 
shown in FIG. 13B, the picture selection unit 213 se- 
lects, as the second reference picture forthecun^ent pic- 
ture to be coded in direct mode, any one of the subse- 
quent pictures of the current block to be coded, I.e. any 
one of the picture with the picture number "15", the pic- 
ture with the picture number "13", and the picture with 
the picture number "11", as In the above case. Then, the 
picture selection unit 21 3 specifies the second reference 
index rRI2 indicating such selected picture based on the 
remapping infomnation, and outputs it. 
[0240] Accordingly, in the present embodiment, it Is 
possible to improve coding efficiency, even when rem- 
apping takes place, since a picture that comes after the 
current picture to be coded in display order Is used as 
the second reference picture at the time of coding In di- 
rect mode, as In the case of the first embodiment. 
[0241] The variable length coding unit 203 perfomns 
variable length coding on the coded residual signals ER, 
the first reference indices RI1 , the second reference in- 
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dices RI2. the first motion vectors M VI , the second mo- 
tion vectors MV2, the direct mode scaling coefficients 
SP, the type information PT, and the remapping informa- 
tion, as well as on each second reference index rRI2 

5 indicating a second reference picture selected by the 
picture selection unit 213, and outputs the resultant as 
a bit stream BS1. In the case where the direct mode 
processing unit 110 does not use the direct mode scal- 
ing coefficients SP but display time infomnation. the var- 

10 iable length coding unit 203 perfomns variable I ength 
coding on the display time information used for the 
above-perfomned scaling without variable length coding 
the direct mode scaling coefficients SP, and incorpo- 
rates the variable length -coded display time information 

IS into the bit stream BS1 

[0242] FIG. 1 8 is a diagram showing the structure of 
a header of the bit stream BS1 . 

[0243] The bit stream BS1 is structured on a pictu re- 
by-picture basis, where Information about the header 41 

20 and blocks is Included on a picture-by-picture basis, as 
in the case of the bit stream BS in the first embodiment. 
[0244] Here, the header 41 Included In the bit stream 
BS1 on a picture-by-picture basis is different from the 
header 21 Included in the bit stream BS on a picture-by- 

25 picture basis in the first embodiment. 

[0245] In other words, each header 41 In the bit 
stream BS1 includes not only the first time difference 
information 24, the second time difference information 
25, the time difference infonDatlon 26, and the coded 

30 remapping infomiation 27, but also second index Infor- 
mation 42 obtained by coding the above-described sec- 
ond reference index rRI2 outputted from the picture se- 
lection unit 213. 

[0246] Note that, as described above, the same value 

35 is used as a second reference Index rR12 for all blocks 
within the current picture to be coded to which direct 
mode shall be applied. Therefore, a second reference 
index rRI2 shall be coded only once per picture. Also, 
the picture decoding apparatus that decodes the bit 

•^0 stream BS1 can uniquely select such second reference 
picture used for direct mode, by obtaining such second 
reference index rRI2 and the remapping information. 
[0247] Also, when not a direct mode scaling coeffi- 
cient SP but display time Infomiation is used for 

45 processing perfomned by the direct mode processing 
unit 110, infomnation obtained by coding the display time 
information of the picture Is included in the header 41 
instead of the first time difference information 24 and the 
second time difference infomnation 25, as well as the 

so time difference information 26. 

[0248] FIG. 1 9 Is a flowchart showing the operation of 
coding a picture in direct mode In the present embodi- 
ment. 

[0249] First, the picture coding apparatus 200 causes 
55 the picture selection unit 213 to select any one of the 
pictures located after the current picture to be coded in 
display order, as the second reference picture (Step 
S200). 
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[0250] Next, the picture coding apparatus 200 causes 
the vector buffer 1 1 4 to specify, as a scaling vector rMV, 
the first motion vector l\^V1 of a blocl( that is co-located 
with the current block and included in the second refer- 
ence picture selected in the above manner, and to spec- 
ify, as a first reference picture, a picture referred to by 
such first motion vector MV1 (Step S202). 
[0251] Then, the picture coding apparatus 200 gener- 
ates a predictive image in direct mode through the direct 
mode processing unit 110. the multi frame buffer 107, 
and the pixel interpolation unit 106, utilizing the second 
reference picture selected In the above manner as well 
as the first reference picture and the scaling vector rMV 
specified in the above manner (Step S204). 
[0252] After generating the predictive image, the pic- 
ture coding apparatus 200 generates a prediction error 
from the current block to be coded and the predictive 
image (Step S206), and then codes such generated pre- 
diction error, the second reference index rRI2 indicating 
the second reference picture, and the remapping infor- 
mation (Step S208). 

[0253] As described above, in the present embodi- 
ment, any one of the reference pictures located after the 
current picture to be coded in display order is selected 
as the second reference picture in direct mode, and the 
second reference index rRI2 of such selected reference 
picture is coded and stored into the bit stream BS1 . 
[0254] In the first embodiment, a picture with the 
smallest second reference index is selected as a second 
reference picture used for direct mode, from among the 
pictures located after the current picture to be coded in 
display order. 

[0255] Stated another way, in the third embodiment, 
as in the case of the first embodiment, it is possible to 
improve coding efficiency by selecting, as the second 
reference picture in direct mode, a picture that comes 
after the current picture to be coded In display order. 
[0256] The difference between the first embodiment 
and the third embodiment is as described below. In the 
first embodiment, a second reference picture in direct 
mode is detemnlned mutually and uniquely between the 
picture coding apparatus 100 and the picture decoding 
apparatus 150 by having the picture coding apparatus 
100 hold the procedure of determining a second refer- 
ence picture in direct mode as well as by having the pic- 
ture decoding apparatus 150 in the second embodiment 
hold the same procedure. In the third embodiment, on 
the other hand, a second reference picture in direct 
mode Is determined mutually and uniquely between the 
picture coding apparatus 200 and the picture decoding 
apparatus by incorporating into a bit stream information 
for uniquely determining such second reference picture 
and by having the picture coding apparatus 200 and the 
picture decoding apparatus that decodes such bit 
stream perform processing based on such infomriation. 
[0257] As described above, according to the present 
embodiment, it is possible to ensure consistency be- 
tween coding processing and decoding processing by 



coding Infomiatlon to identify a second reference picture 
selected in coding processing. 

[0258] Note that in the present embodiment, any one 
of the pictures located after the current picture to be cod- 

s ed in display order is selected as a s econd reference 
picture, but a picture with the smallest second reference 
index may be selected as a second reference picture 
from among the subsequent pictures in display order, 
as in the case of the first embodiment. Furthermore, the 

10 picture selection unit 21 3 may select the first reference 
picture of a current picture to be coded used in direct 
mode. In this case, the first reference Index rR1 1 and the 
second reference index rRI2 shall be coded to be incor- 
porated into the header 41 shown in FIG. 5. Moreover, 

IS the picture decoding apparatus that decodes the bit 
stream BS1 can identify the first reference picture and 
the second reference picture used in direct mode by ob- 
taining thelrfirst reference Index rRII and second refer- 
ence index rRI2 from the bit stream BS1 . 

20 [0259] What is more, according to the format of the bit 
stream BS1 in the present embodiment, the second in- 
dex infomnation 42 obtained by coding a second refer- 
ence index rRI2 is included in the header 41 on a picture- 
by-plcture basis. However, the second reference Infer- 

25 mation 42 may be stored, for example, in ( i ) the header 
of a coding unit smaller than a picture such as an MPEG 
slice, ( ii ) the header of a coding unit made up of plural 
pictures such as a GOP of MPEG, or ( iii ) the. header 
of the whole sequence. Moreover, in the case where a 

30 first reference picture is selected in the above-described 
manner, information obtained by coding a first reference 
index rRII indicating such first reference picture shall 
be stored in a header described above. 



[0260] The following describes the picture decoding 
apparatus according to the fourth embodiment of the 
present invention with reference to the figures. 
40 [0261] FIG. 20 is a block diagram showing the picture 
decoding apparatus according to the fourth embodi- 
ment. 

[0262] Such picture decoding apparatus 250, which 
decodes the bit stream BS1 outputted from the picture 

45 coding apparatus 200 in the third embodiment, is com- 
prised of a variable length decoding unit 251 , the image 
decoding unit 152, the adder 153, the switches 155 and 
156, the direct mode processing unit 154, the pixel in- 
terpolation unit 157, the multi frame buffer 158, and the 

so vector buffer 160. 

[0263] Here, the image decoding unit 152, the adder 
1 53, the switches 1 55 and 1 56, the direct mode process- 
ing unit 154, the pixel interpolation unit 157, the multi 
frame buffer 1 58, and the vector buffer 1 60 of the image 

55 decoding unit 250 respectively have the same functions 
as those of the picture decoding apparatus 1 50 in the 
second embodiment. 

[0264] In other words, the present embodiment is 
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characterized by the function of the variable length de- 
coding unit 251 . 

[0265] The variable length decoding unit 251 obtains 
the bit stream BS1 from outside the picture decoding 
apparatus 250, performs variable length decoding on s 
such bit stream BS 1 , and outputs the coded residual sig- 
nals ER, the motion vectors MV1 and MV2, the refer- 
ence indices RI1 and RI2, the direct mode scaling coef- 
ficients SP, and the type infomnatlon PT. Here, when the 
bit stream BS1 does not Include coded direct mode seal- io 
ing coefficients SP but coded display time information, 
the variable length decoding unit 251 outputs the display 
time infomiation instead of the direct mode scaling co- 
efficients SP. 

[0266] Furthenmore, the variable length decoding unit is 
251 decodes the second index infonmation 42 included 
in each header 41 in the bit stream BS1 , and outputs 
the second reference indices rRI2 to the vector buffer 
1 60 and the direct mode processing unit 1 54. Moreover, 
when each header 41 includes coded remapping infer- 20 
mation 27, the variable length decoding unit 251 de- 
codes the coded remapping information 27, so as to 
generate remapping information. Then, the variable 
length decoding unit 251 causes the multi frame buffer 
1 58 to specify a picture indicated by each first reference 25 
index RI1 (rRM) and second reference index RI2 (rRI2), 
based on the remapping infomiation. 
[0267] The vector buffer 1 60 obtains, from the varia- 
ble length decoding unit 251 , the second reference in- 
dex rRI2, and specifies, as the second reference picture 30 
of the current picture to be decoded, a picture indicated 
by such second reference index rRI2. Then, the vector 
buffer 1 60 outputs, as a scaling vector rMV, the first mo- 
tion vector MV1 of a predetenmined block included in the 
second reference picture, and outputs the first reference 35 
index RM indicating a picture referred to by such scaling 
vector rIVIV as a first reference index rRM . 
[0268] As described above, in the present embodi- 
ment, the second reference picture used in direct mode 
is uniquely determined from among the pictures located 40 
after the current picture to be decoded in display mode, 
according to each second index infomiation 42 and 
each coded remapping infonmation 27 included in the 
bit stream BS1 . 

[0269] Here, a description is given of the operation of ^5 
decoding a picture In direct mode in the present embod- 
iment, 

[0270] FiG. 21 is a flowchart showing the operation of 
decoding a picture in direct mode in the present embod- 
iment, 50 
[0271] First, the picture decoding apparatus 250 
causes the variable length decoding unit 251 to perfomri 
variable length decoding on the bit stream BS1, so as 
to obtain the second reference index rRI2 (Step S250). 
When the bit stream BS1 includes the coded remapping 55 
infonnation 27. the picture decoding apparatus 250 ob- 
tains remapping Infonmation by decoding such coded 
remapping information 27. 
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[0272] Next, the picture decoding apparatus 250 
causes the vector buffer 1 60 to specify, as a scaling vec- 
tor rMV of the current block to be decoded, the first mo- 
tion vector of a block that Is co-located with such current 
block and Included in the second reference picture indi- 
cated by the second reference index rRI2, and to spec- 
ify, as the first reference picture of the current picture to 
be decoded, a picture referred to by such first motion 
vector MV1 (Step S252). 

[0273] Then, the picture decoding apparatus 250 
causes the direct mode processing unit 154, the multi 
frame buffer 158, and the pixel interpolation unit 157 to 
generate a predictive image in direct mode, utilizing the 
second reference picture indicated by the second refer- 
ence Index rRI2 obtained in the above manner, as well 
as the first reference picture and the scaling vector rMV 
specified in the above manner (Step S254). 
[0274] Then, the picture decoding apparatus 250 
adds such predictive image to residual data generated 
by decoding the bit stream BS1 (Step S256). 
[0275] As described above, in the present embodi- 
ment, since a second reference picture in direct mode 
is specified based on each second index information 42 
and remapping infonmation 41 included in the bit stream 
BS1 , it is possible to correctly decode the bit stream BS1 
transmitted from the picture coding apparatus 200 ac- 
cording to the third embodiment. Moreover, it is possible 
to obtain the same effect as is described in the third em- 
bodiment. 

(Fifth Embodiment) 

[0276] The following describes the picture coding ap- 
paratus according to the fifth embodiment of the present 
invention with reference to the figures. 
[0277] FIG. 22 is a block diagram showing the picture 
coding apparatus according to the fifth embodiment. 
[0278] Such picture coding apparatus 300, which ob- 
tains a picture signal img with the contents representing 
pictures and codes such picture signal Img on a block- 
by-block basis, is comprised of the subtracter 101, the 
image decoding unit 104, the switches 111 and 112, the 
pixel interpolation unit 106. the direct mode processing 
unit 1 1 0, the vector buffer 1 1 4, as in the case of the pic- 
ture coding apparatus 1 00 according to the first embod- 
iment, and further includes a management unit 304, a 
motion estimation unit 302, a prediction type selection 
unit 301 , a multi frame buffer 303, and a variable length 
coding unit 305. 

[0279] Here, the subtracter 1 01 , the Image decoding 
unit 104, the switches 111 and 112, the pixel interpola- 
tion unit 106, the direct mode processing unit 110, and 
the vector buffer 1 1 4 in the picture coding apparatus 300 
respectively have the same functions as those of the pic- 
ture coding apparatus 100 in the first embodiment. 
[0280] Here, the vector buffer 114 obtains a second 
reference index rRI2 that always Indicates 0, and out- 
puts the scaling vector rMV and the first reference index 
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rR1 1 corresponding to such second reference index rRI2 
"0". 

[0281] Moreover, the multi frame buffer 303 in the 
present embodiment has the function equivalent to that 
of the multi frame buffer 1 07 in the first and third embod- 
iments, but information which it stores is under the man- 
agement of the management unit 304. 
[0282] Furthemnore, the multi frame buffer 303 has a 
short-term memory and a long-term memory, where pic- 
tures are appropriately stored into either the short-tenn 
memory or the long-term memory. 
[0283] The short-tenn memory Is a first-in -first- out 
(FIFO) memory. When a new signal is stored into this 
short-term memory, its storage contents are deleted in 
decreasing order of time stored in the memory, so that 
a certain number of the latest pictures are always stored 
in such short-term memory as reference pictures. 
[0284] The iong-tenm memory is a random access 
memory capable of storing reference pictures in an ar- 
bitrary area as well as of reading out reference pictures 
stored in an arbitrary area. Such long-term memory, 
which stores pictures to be referred to for a longer period 
of time (e.g. background pictures and pictures before 
scene changes occur), stores pictures equivalent to a 
longer period of time than those stored in the short-tenn 
memory. Also, pictures are stored into the long-term 
memory by moving pictures stored In the short-term 
memory Into the long-term memory. 
[0285] The management unit 304 manages informa- 
tion stored in the multi frame buffer 303 in the above 
manner, and causes remapping to take place when a 
management result indicates the need for remapping, 
in other words, the management unit 304 generates 
remapping information, and outputs such remapping in- 
fomnation to the prediction type selection unit 301 , the 
motion estimation unit 302, the variable length coding 
unit 305, and the multi frame buffer 303. 
[0286] For example, when judging that there has been 
a change in the scene (hereinafter referred to as "scene 
change") based on a result of obtaining scene change 
infonnation to be descried below, the management unit 
304 generates remapping information. 
[0287] The prediction type selection unit 301 , which 
has the same function as that of the prediction type se- 
lection unit 1 01 in the first and third embodiments, spec- 
ifies the first reference index and the second reference 
index assigned to each reference picture based on the 
remapping information, when obtaining such remapping 
infonnation from the management unit 304. 
[0288] The motion estimation unit 302. which has the 
same function as that of the motion estimation unit 1 02 
in the first and third embodiments, specifies the first ref- 
erence index and the second reference index assigned 
to each reference picture based on the remapping infor- 
mation, when obtaining such remapping information 
from the management unit 304. 
[0289] Stated another way, when not obtaining rema- 
pping infonnation from the management unit 304, the 



prediction type selection unit 301 and the motion esti- 
mation unit 302 assign numerical values to the respec- 
tive reference pictures preceding and subsequent to the 
cun-ent picture to be coded, according to the default or- 
5 der, i.e. the order shown in FIG. 13A, so as to specify 
the first reference indices and the second reference in- 
dices. Meanwhile, when obtaining remapping informa- 
tion from the management unit 304, the prediction type 
selection unit 301 and the motion estimation unit 302 
10 assign numerical values to the respective reference pic- 
tures preceding and subsequent to the current picture 
to be coded, according to the order to be determined by 
taking into account the remapping information to the de- 
fault order, so as to specify the first reference indices 
15 and the second reference indices. 

[0290] The variable length coding unit 305 performs 
variable length coding on the coded residual signals ER, 
the first reference Indices RI1 , the second reference In- 
dices Ri2, the first motion vectors MVI , the second mo- 
tion vectors MV2, the direct mode scaling coefficients 
SP, and the type information PT, so as to output the re- 
sultant as a bit stream BS2, as in the case of the variable 
length coding unit 105 in the first and third embodiments. 
Moreover, when obtaining remapping Information from 
the management unit 304, the variable length coding 
unit 305 codes such remapping infonnation, and incor- 
porates such coded remapping information into each 
header of the bit stream BS2. In the case where display 
time infonnation is used instead of the direct mode scal- 
ing coefficients SP for the processing performed by the 
direct processing unit 110, the variable length coding 
unit 305 performs variable I ength coding on the display 
time infonnation used for the above-performed scaling, 
without variable length coding the direct mode scaling 
coefficients SP, and incorporates such variable length- 
coded display time information into the bit stream BS2. 
[0291] Here, when generating remapping infonna- 
tion, the management unit 304 according to the present 
embodiment generates such remapping infonnation as 
makes first reference indices and second reference in- 
dices assigned to the respective reference pictures sat- 
isfy a certain condition. 

[0292] In the case where there is a scene change 
when the current block is to be coded by means of in- 
terpolative prediction other than direct mode, the man- 
agement unit 304 generates such remapping informa- 
tion as allows the first reference index of 0 to be as- 
signed preferentially to a reference picture stored in the 
long-tenn memory. The judgment on whether a scene 
change occurs or not is made outside the picture coding 
apparatus 300 on the basis of the input picture signal 
Img. so that the result of such judgment shall be provid- 
ed to the management unit 304 as scene change infor- 
mation. Scene change detection methods (judgment 
methods) include the following method: determine an 
absolute value difference between each of pixels co-lo- 
cated with each other in temporally successive pictures, 
so as to add absolute difference values of all the pixels 
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in the pictures, and Judge that there is a scene change 
when such addition value exceeds a predetemnined 
threshold. Furthermore, it is possible to code a franne 
(picture) detected as a scene change as an I picture. 
Accordingly, it becomes possible to search out the point 
of a scene change by searching for an I picture. 
[0293] Furthermore, when the current block is coded 
by means of interpolative prediction in direct mode, the 
management unit 304 generates such remapping infor- 
mation as allows the second reference index of 0 to be 
assigned to a reference picture located after such cur- 
rent picture to be coded in display order. 
[0294] Here, the management unit 304 may generate 
remapping information that does not change the second 
reference index "0" of the reference picture which shall 
be assigned such second reference index 0 by default. 
[0295] This enables the second reference index 0 to 
be always assigned to a reference picture that is located 
after the current picture to be coded in display order, 
even when remapping occurs. 

[0296] Referring to FIG. 23, a description is given of 
remapping perfomned by the management unit 304 with 
the above configuration. 

[0297] FIG. 23 is a diagram explaining remapping per- 
formed by the management unit 304. 
[0298] (a) in FIG. 23 is a diagram showing the ar- 
rangement of the current picture to be coded and refer- 
ence pictures in display order. 

[0299] When a current picture TP, which is a B picture, 
is coded, any of the P pictures preceding or subsequent 
to such current picture to be coded TP, e.g. reference 
pictures RP1 . RP2, RP3. RP4 and RP5. is used. More- 
over, the reference pictures RP2, RP3, RP4, and RP6, 
for example, which are located near the current picture 
TP are stored in the short-temn memory in the multi 
frame buffer 303, whereas the reference picture RP1 lo- 
cated far away from the current picture TP is stored In 
the long-tenn memory in the multi frame buffer 303. 
[0300] I n order to identify the reference pictures R P1 , 
RP2, RP3. RP4 and RP5, first reference indices and 
second reference Indices are assigned to these pictures 
as shown in (b) in FIG. 23 by default. 
[0301] As (b) in FIG. 23 shows, the reference picture 
RP1 is assigned the first reference index "4" and the 
second reference index "4", the reference picture RP2 
is assigned the first reference index "1" and the second 
reference Index "3", the reference picture RP3 is as- 
signed the first reference index "0" and the second ref- 
erence index "2", the reference picture RP4 is assigned 
the first reference index "2" and the second reference 
index "0", and the reference picture RP5 is assigned the 
first reference index "3" and the second reference index 
"1". 

[0302] Here, in the case where the current picture TP 
is to be coded by means of interpolative prediction other 
than in direct mode, the management unit 304 gener- 
ates remapping infonmatlon that allows the first refer- 
ence index 0 to be assigned from a reference picture 



stored in the long-term memory, when making the fol- 
lowing judgments: the scene A similar to the scene A' 
that includes the current picture TP is older than the 
scene B which is not similar to the scene A'; and the 
5 reference picture that is located temporally and imme- 
diately before the scene A* (being the reference picture 
RP3 in this case) is included in the scene B which is not 
similar to the scene A*, i.e. a scene change occurs after 
the reference picture that is located temporally and im- 
10 mediately before the scene A'. 

[0303] In other words, as shown in (c) in FIG. 23, the 
first reference index 0 is assigned to the reference pic- 
ture RP1 stored in the long-term memory. 
[0304] In the present embodiment, as described 
IS above, when a scene change occurs at the time of cod- 
ing the cun-ent picture by means of interpolative predic- 
tion other than in direct mode, remapping information is 
generated that enables the first reference Index 0 to be 
preferentially assigned to a reference picture stored in 
the long-temri memory. Accordingly, the first reference 
picture with the first reference Index "0" used for inter- 
polative prediction has picture information much similar 
to the current picture to be coded. This consequently 
contributes to the improvement of coding efficiency. 
[0305] Stated another way, as shown in (b) in FIG. 23, 
in the case where such remapping information is not to 
be generated, the reference picture RP3 indicated by 
the first reference index "0" shall be used for coding the 
current picture TP. However, since the reference picture 
RP3 is dissimilar to the cun^ent picture to be coded TP 
because the reference picture RP3 is included in a dif- 
ferent scene, coding efficiency becomes lowered. 
[0306] Meanwhile, when the current picture TP is cod- 
ed in direct mode, the management unit 304 generates 
remapping Infonnation as shown in (d) in FIG. 23 that 
allows the second reference index 0 to be assigned to 
a reference picture that comes after the current picture 
to be coded TP in display order, e.g. the reference pic- 
ture RP5. Furthermore, the management unit 304 may 
generate remapping Infonnation as shown In (b) In FIG. 
23 that does not allow the second reference index "0". 
which shall be assigned to the reference picture RP4 by 
default, to be assigned to another reference picture. 
[0307] As described above, in the present embodi- 
ment, when the current block is coded by interpolative 
prediction in direct mode, the second reference index 0 
Is assigned to a reference picture located after the cur- 
rent picture to be coded in display order, so as to use 
such reference picture with the second reference index 
"0" as the second reference picture. Accordingly, it be- 
comes possible to improve coding efficiency. 
[0308] FIG. 24 is a flowchart showing an example op- 
eration performed in the present embodiment. 
[0309] First, the picture coding apparatus 300 judges 
whether or not there Is a scene change, according to 
whether or not the management unit 304 has obtained 
the above-mentioned scene change information (Step 
S300). 
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[0310] Here, when judging that there is a scene 
change (Y in Step S300), the picture coding apparatus 
300 perfonms remapping in a manner in which the fol- 
lowing conditions are satisfied (Step S302): the first ref- 
erence index 0 is assigned to a reference picture stored 
in the long-term memory; and the second reference in- 
dex 0 is assigned to any one of the reference pictures 
located after the current picture to be coded. Meanwhile, 
when Judging that there is no scene change (N in Step 
S300), the picture coding apparatus 300 assigns first 
reference indices and second reference indices accord* 
ing to the defauft, without performing remapping (Step 
S304). 

[0311] Next, the picture coding apparatus 300 speci- 
fies the first reference picture and the second reference 
picture based on the first reference indices and second 
reference indices assigned in the above manner, and 
generates a predictive Image based on these reference 
pictures (Step S306). 

[0312] Subsequently, the picture coding apparatus 
300 generates a prediction error from the predictive im- 
age (Step S308), and codes it (Step S310). 

(Variation) 

[031 3] Here, a description is given of a variation of the 
management unit according to the present embodiment. 
[0314] FIG. 25 is a diagram showing a configuration 
of a picture coding apparatus 300' according to the var- 
iation of the present embodiment. 
[0315] The picture coding apparatus 300' according 
to the variation is comprised of the image coding unit 
102, the image decoding unit 104, a management unit 
304' and others, as in the case of the aforementioned 
picture coding apparatus 300. 

[0316] Here, in the present embodiment, the second 
reference index 0 shall be assigned to a reference pic- 
ture that Is located after the current picture to be coded 
in display order, when such current picture is coded by 
means of interpolative prediction in direct mode. Stated 
another way, the management unit 304' prohibits the as- 
signment of the second reference index 0 to any refer- 
ence pictures that have been located before the cun-ent 
picture in display order. As a result, no first motion vec- 
tors used for coding blocks in a reference picture that 
has been located before a current picture in display or- 
der, shall be used as scaling vectors. 
[0317] Thus, the present variation can improve coding 
efficiency by omitting processing for storing. Into the 
vector buffer 114, the first motion vectors of blocks in a 
reference picture that has been located before a current 
picture in display order for use in the coding of such cur- 
rent picture. In other words, in the present variation, the 
management unit 304' in the picture coding apparatus 
300* deletes, from the vector buffer 1 1 4, the first motion 
vectors of a reference picture that has been located be- 
fore the current picture In display order. 
[0318] This prevents motion vectors not necessary for 



coding from being stored in the vector buffer 114, and 
therefore reduces the amount of memory of the vector 
buffer 114. 

5 (Sixth Embodiment) 

[0319] The following describes the picture decoding 
apparatus according to the sixth embodiment of the 
present invention with reference to the figures. 
10 [0320] FIG. 26 is a block diagram showing the picture 
decoding apparatus according to the sixth embodiment. 
[0321] Such picture decoding apparatus 350, which 
decodes the bit stream BS2 outputted from the picture 
coding apparatus 300' according to the variation of the 
IS fifth embodiment, is comprised of the variable length de- 
coding unit 151 , the image decoding unit 152, the adder 
153, the switches 155 and 156. the direct mode process- 
ing unit 154, the pixel interpolation unit 157, the multi 
frame buffer 158, and the vector buffer 160, as in the 
case of the second embodiment, and further includes a 
management unit 354. 

[0322] The picture decoding apparatus 350 has the 
same configuration as the one described in the second 
embodiment in which the picture selection unit 159 out- 
puts, to the vector buffer 1 60. the second reference in- 
dex rRI2 that always Indicate 0. 
[0323] Moreover, the variable length decoding unit 
151 obtains the bit stream BS2, and when such bit 
stream BS2 include remapping infomnation, the variable 
length decoding unit 151 causes the multi frame buffer 
158 to specify a picture indicated by each first reference 
Index RI1 (rRII) and second reference index RI2 (rRI2), 
based on such remapping information. 
[0324] Here, the management unit 354 monitors ref- 
erence pictures stored in the multi frame buffer 158, so 
as to delete, from the vector buffer 160, the first motion 
vectors M VI of a reference picture that has been located 
before the current picture to be decoded in display order. 
[0325] This prevents motion vectors not necessary for 
decoding from being stored in the vector buffer 1 60 also 
in the present embodiment, as in the case of the varia- 
tion of the fifth embodiment, and therefore reduces the 
amount of memory of the vector buffer 1 60. 

(Seventh Embodiment) 

[0326] FIG. 27 is a block diagram showing a picture 
coding apparatus according to the seventh embodi- 
ment. 

[0327] Here, units and signals shown in FIG. 27 which 
are the same as those of the existing picture coding ap- 
paratus 900 shown in FIG. 7 are assigned the same 
numbers as those assigned to the units and signals of 
the picture coding apparatus 900, and therefore no de- 
scription is given of them. 

[0328] The present embodiment is characterized by 
that the first reference picture selection unit 601 selects 
one first reference picture to be commonly used for cod- 
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ing the blocks in the current picture In direct mode. Ac- 
cordingly, a predetermined reference picture shall be 
commonly used as the first reference picture to be re- 
ferred to by scaling vectors In direct mode. 
[0329] FIG. 28 is a conceptual diagram showing direct 
mode according to the seventh embodiment. 
[0330] In FIG, 28, the reference picture RP1 serves 
as the first reference picture to be commonly used in 
direct mode. In other words, the first motion vector MV1 0 
of the block BOO refers to the reference picture RPn, but 
the first motion vector MV01 of the block BO to which 
direct mode Is applied refers to the above-described ref- 
erence picture RP1. 

[0331] More specifically, the motion vector MVO ap- 
proximately derived from the block B1 on the common 
reference picture RP1 is used as a scaling vector, so as 
to derive the motion vectors MV01 and MV02 used for 
coding the block BO In the current picture TP based on 
the scaling of such scaling vector. Stated another way, 
the motion vector MV01 of the block BO on the current 
picture to be coded TP which is a motion vector derived 
from the block RB01 on the picture RP1 and the motion 
vector MV02 of the block RB02 on the picture RP3 are 
determined by using the motion vector MVO as a scaling 
vector. 

[0332] Note that in the present embodiment, the direct 
mode processing unit 910' performs the above-de- 
scribed processing in direct mode. 
[0333] Meanwhile, conventionally, infomriation for 
identifying the reference picture RPn to be refereed to 
by the motion vector MV10 is required for performing 
scaling and such infonnation needs to be stored on the 
memory. 

[0334] Here, referring to FIG. 29. a description is giv- 
en of a method of calculating the motion vectors MV01 
and MV02. 

[0335] FIG. 29 is a flowchart showing the process of 
calculating the motion vectors MVOI and MV02. 
[0336] First, the picture coding apparatus 600 deter- 
mines the block B1 on the first reference picture RP1 
which is co-located with the block Bn refen^ed to by the 
motion vector MV10 of the block BOO (Step S1). 
[0337] Next, the picture coding apparatus 600 re- 
gards the motion vector MVO extending from the block 
BOO to the block B1 as a scaling vector (Step S2). 
[0338] Then, the picture coding apparatus 600 deter- 
mines the motion vectors MVOI and MV02 from such 
scaling vector MVO (Step S3). 

[0339] As described above, by using one picture as 
the first reference picture to be commonly used in direct 
mode regardless of a reference picture referred to by 
the motion vector MV10, it becomes not necessary for 
the vector buffer 914 to store the picture number of a 
reference picture referred to by the motion vector MV1 0 
(being the picture number of the reference picture RPn 
in FIG. 28). Instead, as shown in FIG. 27, the first refer- 
ence picture selection unit 601 selects a first reference 
picture to be commonly used In direct mode, and outputs 



the first reference index rRII of such first reference pic- 
ture. Note that an example method of selecting a first 
reference picture to be commonly used in direct mode 
is to select a reference picture which comes before the 
5 current picture to be coded in display order and which 
is assigned the smallest first reference index value. 
[0340] As described above, according to the present 
embodiment, by using one first reference picture as the 
picture to be commonly used for all the blocks in a pic- 
10 ture to be coded in direct mode, there is no need to store 
in the memory information used to identify a reference 
picture referred to by a scaling vector. In other words, it 
becomes possible to reduce the amount of memory. Fur- 
thermore, by using a single first reference picture as the 
IS picture to be commonly used for all the blocks in a pic- 
ture to be coded in direct mode, it becomes possible to 
simplify coding processing and therefore to improve 
coding efficiency. Note that infomiatton used to Identify 
a picture referred to by a first motion vector is, for ex- 
ample, Its picture number shown In FIG. 1 3, but any oth- 
er information may serve as such Identification infomria- 
tion as long as a picture referred to by the motion vector 
can be uniquely determined by such information. 
[0341] Note that in the present embodiment, the pic- 
ture with the smallest first reference index among pic- 
tures located before the current picture to be coded is 
selected as the first reference picture to be commonly 
used in direct mode. However, the reference picture that 
is closest to the current picture In display order among 
the pictures preceding such current picture may be se- 
lected as the above first reference picture. 
[0342] Also, in the present embodiment, by using a 
preceding picture of the cun-ent picture as at least one 
of the reference pictures referred to by such current pic- 
ture to be coded, which is a B picture, there arises no 
possibility that only pictures located after the current pic- 
ture to be coded will be used as reference pictures. Ac- 
cordingly, it becomes possible to raise the possibility of 
improving coding efficiency. 

[0343] Furthemiore, in the present embodiment, a 
second reference picture may be selected in a manner 
equivalent to that of the first embodiment. In this case, 
coding efficiency will be improved as a result of combing 
the coding method presented in the first embodiment 
and the coding method presented in the present embod- 
iment. 

[0344] Moreover, In the present embodiment, the di- 
rect mode processing unit 910' performs scaling on a 
scaling vector ri\4V by use of a direct mode scaling co- 
efficient SP However, display time information may be 
used to perform scaling on the scaling vector rMV, as in 
the case of the other embodiments. In such case, refer- 
ring to FIG. 28, not the motion vector MVO but the motion 
vector MV10 shall be used as a scaling vector. Stated 
another way, the first motion vector MVOI of the current 
block BO is determined by scaling the motion vector 
MV10 according to the ratio of the difference In the dis- 
play times between the picture TP and the picture RPn 
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with respect to the difference in the display times be- 
tween the picture RP3 and the picture RPn, and the sec- 
ond motion vector IS^V02 of the current blocic BO is de- 
tennined by scaling the motion vector MV10 according 
to the ratio of the difference in the display times between 
the picture TP and the picture RP3 with respect to the 
difference between the display times in the picture RP3 
and the picture RPn. 

(Eighth Embodiment) 

[0345] FIG. 30 is a block diagram showing a picture 
decoding apparatus according to the eighth embodi- 
ment. 

[0346] Here, units and signals shown In FIG. 30 which 
are the same as those of the existing picture decoding 
apparatus 950 shown in FIG. 9 are assigned the same 
numbers as those assigned to the units and signals of 
the picture decoding apparatus 950, and no description 
is given of them. 

[0347] The picture decoding apparatus 650 according 
to the present embodiment decodes the bit stream BSO 
outputted by the picture coding apparatus 600 de- 
scribed in the seventh embodiment. 
[0348] A first reference picture selection unit 651 se- 
lects a first reference picture in a manner which is the 
same as that of the first reference picture selection unit 
601 shown in FIG. 27. Then, the first reference picture 
selection unit 651 outputs, to the direct mode processing 
unit 954', the first reference index rRII of such selected 
first reference picture. 

[0349] Subsequently, the direct mode processing unit 
954' performs the processing in direct mode shown in 
FIG. 28. 

[0350] As described above, by using one first refer- 
ence picture which shall be commonly used for all the 
bloclcs in a picture subject to direct mode coding, It be- 
comes unnecessary to hold InfonDation used to identify 
reference pictures for direct mode. This also contributes 
to the reduction in the amount of memory of the vector 
buffer 960 compared to the conventional cases. 
[0351] As described above, according to the present 
embodiment, it Is possible to con-ectly decode the bit 
stream BSO which has been coded by the picture coding 
apparatus 600 in the seventh embodiment. Moreover, it 
is possible to obtain the same effect as is described in 
the seventh embodiment. 

(Ninth Embodiment) 

[0352] FIG. 31 is a block diagram showing a picture 
coding apparatus according to the ninth embodiment. 
[0353] Here, units and signals shown in FIG. 31 which 
are the same as those of the existing picture coding ap- 
paratus 900 shown in FIG. 7 are assigned the same 
numbers as those assigned to the units and signals of 
the picture coding apparatus 900, and therefore no de- 
scription Is given of them. 



[0354] In the seventh embodiment, the first reference 
picture to be commonly used in direct mode is selected 
by the same single method which has been predeter- 
mined by the picture coding apparatus 600 and the pic- 

5 ture decoding apparatus 650, and information indicating 
such selected first reference picture shall not be stored 
in the bit stream BSO. In the present embodiment, how- 
ever, the first reference index of the selected first refer- 
ence picture shall be stored in a bit stream. 

10 [0355] In other words, the ninth embodiment is differ- 
ent from the seventh embodiment in the point described 
below. In the seventh embodiment, the first reference 
picture in direct mode is uniquely determined by having 
the picture coding apparatus 600 hold the procedure for 

15 uniquely determining the first reference picture in direct 
mode as well as by having the picture decoding appa- 
ratus 650 in the eighth embodiment hold the same pro- 
cedure. In the ninth embodiment, on the other hand, by 
incorporating, into a bit stream, information used for 

20 uniquely determining the first reference picture in direct 
mode, such first reference picture in direct mode is mu- 
tually and uniquely detennined by the picture decoding 
apparatus that decodes such bit stream and by the pic- 
ture coding apparatus 700. 

25 [0356] The first reference picture selection unit 701 
selects a first reference picture for direct mode from 
among reference pictures, and outputs the first refer- 
ence index rRI1 of such selected reference picture to 
the direct mode processing unit 910' and the variable 

30 length coding unit 903. 

[0357] The variable length coding unit 903 performs 
variable length coding on each coded residual signal 
ER, first reference Index RI1, second reference index 
RI2, first motion vector MV1 , second motion vector l\^V2, 

35 direct mode scaling coefficient SP, type information PT, 
and first reference index rRII which is outputted by the 
first reference picture selection unit 701 , and outputs the 
resultant as a bit stream BS3. In the case where display 
time information is used by the direct mode processing 

40 unit 910' instead of the direct mode scaling coefficient 
SP, the variable length coding unit 903 performs variable 
length coding on the display time information used for 
the above-performed seating without variable length 
coding the direct mode scaling coefficient SP, and incor- 

45 porates the variable length-coded display time informa- 
tion into the bit stream BS3. 

[0358] FIG. 32 shows the f omriat of the bit stream BS3 
according to the ninth embodiment. 
[0359] The header is the only difference between the 
50 format of the bit stream BS3 and the fonnat of the bit 
stream BSO of the existing picture coding apparatus 900 
shown in FIG. 6. 

[0360] The same value is used as a first reference in- 
dex rRII for all blocks within a picture to which direct 
55 mode shall be applied. Therefore, it is enough to code 
the first reference index rRII only once per picture. Such 
being the case, the variable length coding unit 903 
codes the first reference index rRII , so as to incorporate 
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such coded first reference index rRil into each header 
61 of the bit stream BS3 as first index infonnation 52. 
Note that the header 51 may include display time infor- 
mation instead of the first time difference infonnation 4 
and the second time difference information 5, as well as 
the time difference information 6 shown in FIG. 32. The 
picture decoding apparatus that decodes the bit stream 
BS3 can specify the first reference picture for direct 
mode by obtaining, from the bit stream BS3, the first ref- 
erence index rRI1 indicated by the first index information 
52. 

[0361] As described above, according to the present 
embodiment, by using the common first reference pic- 
ture for coding all the blocks within a picture which shall 
be coded in direct mode, it becomes unnecessary to 
store information used to identify a reference picture ref- 
ereed to by a scaling vector. Stated another way, it be- 
comes possible to reduce the amount of memory of the 
vector buffer 914 compared with the conventional cas- 
es. IVIoreover, it is also possible to simplify coding 
processing by using the common first reference picture 
for coding all the blocks within a picture which shall be 
coded in direct mode. Also, it is possible to ensure con- 
sistency between coding processing and decoding 
processing by coding infonnation indicating the first ref- 
erence picture used for direct mode and by incorporat- 
ing such coded information into the bit stream BS3. 
[0362] Note that it Is also possible to select a picture 
with the smallest first reference index among pictures 
that are located before the current picture to be coded 
in display order, as the first reference picture to be com- 
monly used in direct mode. 

[0363] Also, in the present embodiment, by using a 
preceding picture of the current picture as at least one 
of the reference pictures referred to by such current pic- 
ture to be coded, which Is a B picture, there arises no 
possibility that only pictures located after the current pic- 
ture to be coded will be used as reference pictures. Ac- 
cordingly, it becomes possible to improve coding effi- 
ciency. 

[0364] Furthennore, in the present embodiment, a 
second reference picture may be selected In a manner 
equivalent to that of the third embodiment. In this case, 
the first Index infonnation 52 obtained by coding the first 
reference index rRM and the second index information 
obtained by coding the second reference index rRI2 are 
incorporated into the header 51 shown In FIG. 32. More- 
over, the picture decoding apparatus that decodes the 
bit stream BS3 can specify the first reference picture and 
second reference picture used for direct mode by ob- 
taining the first reference index rRII indicated by such 
first index information 52 and the second reference in- 
dex rRI2 indicated by such second index infomnation. 
[0365] What is more, according to the format of the bit 
stream BS3 in the present embodiment, the first index 
infonnation 52 obtained by coding the first reference in- 
dex rRI1 is included in the header 51 on a picture-by- 
picture basis. However, the first index Infonnation 52 
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may be stored, for example, in ( t ) the header of a cod- 
ing unit smaller than a picture such as an MPEG slice, 
( ii ) the header of a coding unit made up of plural pic- 
tures such a GOP (Group of Picture) of MPEG, or ( iil ) 
5 the header of the whole sequence. 

(Tenth Embodiment) 

[0366] FIG. 33 Is a block diagram showing a picture 
10 decoding apparatus according to the tenth embodiment. 
[0367] Here, units and signals shown in FIG. 33 which 
are the same as those of the existing picture decoding 
apparatus 950 shown In FIG. 9 are assigned the same 
numbers as those assigned to the units and signals of 
15 the picture decoding apparatus 950. and no description 
is given of them. 

[0368] The picture decoding apparatus 750 according 
to the present embodiment decodes the bit stream BS3 
outputted by the picture coding apparatus 700 de- 
scribed in the ninth embodiment. I.e. the bit stream that 
includes infonnation indicating the first reference index 
of the first reference picture to be commonly used for 
direct mode. 

[0369] The variable length decoding unit 951 obtains 
the bit stream BS3 so as to perfonn variable length de- 
coding on it, and outputs the coded residual signals ER, 
the first reference indices RI1 , the second reference in- 
dices RI2, the first motion vectors MV1 , the second mo- 
tion vectors MV2, the direct mode scaling coefficients 
SP, the type information PT, and the first reference index 
rRM for direct mode. Here, when the bit stream BS3 in- 
cludes not the coded direct mode scaling coefficients 
SP but coded display time information, the variable 
length decoding unit 951 outputs the display time infor- 
mation instead of the direct mode scaling coefficients 
SP 

[0370] The first reference index rRM for direct mode 
indicates the first reference picture to be commonly 
used for direct mode. Thus, there is no need for the vec- 
tor buffer 960 to hold rnfomnatlon for Identifying refer- 
ence pictures used for direct mode, resulting in the re- 
duction in the amount of memory of the vector buffer 
960. 

[0371] As described above, according to the present 
embodiment, it is possible to correctly decode the bit 
stream BS3 outputted by the picture coding apparatus 
700 in the ninth embodiment. Furthermore, by using the 
same reference picture for all the blocks within a picture 
which are coded In direct mode. It becomes unneces- 
sary to store, on the memory, Infonnation for identifying 
a reference picture referred to by a scaling vector. In 
other words, it becomes possible to reduce the amount 
of memory of the vector buffer 960 compared with the 
conventional cases. Moreover, It is possible to obtain the 
same effect as described in the ninth embodiment. 
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(Eleventh Embodiment) 

[0372] tf a program for realizing the picture coding 
method or the picture decoding method as shown in 
each of the aforementioned embodiments is recorded 
on a recording medium such as a flexible disk, it be- 
comes possible to easily perform the processing pre- 
sented in each of the above embodiments in an Inde- 
pendent computer system. 

[0373] FIGS. 34A, 34B. and 34C are diagrams ex- 
plaining a recording medium that stores a program for 
realizing the picture coding method and the picture de- 
coding method according to the first to tenth embodi- 
ments. 

[0374] FIG. 34B shows an external view of a flexible 
disl< FD viewed from the front, a schematic cross-sec- 
tional view, and a disk body FD1 , while FIG. 34A illus- 
trates an example physical fomiat of the disk body FD1 
as a recording medium itself. 

[0375] The disk body FD1 is contained in a case F, 
and a plurality of tracks Tr are formed concentrically on 
the surface of the disk body FD1 in the radius direction 
from the periphery, each track being divided into 1 6 sec- 
tors Se In the angular direction. Therefore, in the flexible 
disk FD storing the above-mentioned program, the pic- 
ture coding method or the picture decoding method as 
such program Is recorded in an area allocated for it on 
the disk body FD1 . 

[0376] Meanwhile, FIG. 34C shows the structure for 
recording and reading out the program on and from the 
flexible disk FD. 

.[0377] When the program is recorded on the flexible 
disk FD, the picture coding method or the picture decod- 
ing method as the above program is written by the use 
of the computer system Cs via a flexible disk drive FDD. 
Meanwhile, when the picture coding method or the pic- 
ture decoding method is constructed in the computer 
system Cs through the program on the flexible disk FD, 
the program is read out from the flexible disk FD via the 
flexible disk drive FDD and transferred to the computer 
system Cs. 

[0378] The above explanation is made on the as- 
sumption that a recording medium is a flexible disk FD, 
but an optical disc may also be used. In addition, the 
recording medium is not limited to this, and any other 
medium such as an IC card and a ROM cassette capa- 
ble of recording a program can also be used. 

(Twelfth Embodiment) 

[0379] The following explains applications of the pic- 
ture coding method and the picture decoding method as 
shown in the above embodiments as well as a system 
using them. 

[0380] FIG. 35 is a block diagram showing an overall 
configuration of a content supply system ex100 for re- 
alizing a content distribution service. The area for pro- 
viding a communication service is divided Into cells of 



desired size, and base stations ex107 — ex110, which 
are fixed wireless stations, are placed in the respective 
cells. 

[0381] In this content supply system exIOO, devices 

5 such as a computer ex1 11 , a PDA (Personal Digital As- 
sistant) exi 1 2, a camera exi 1 3, a cellular phone exil 4, 
and a camera-equipped cellular phone ex115 are re- 
spectively connected to the Internet exi 01 via an Inter- 
net service provider exi 02, a telephone network ex1 04, 

10 and the base stations ex1 07 exi 1 0. 

[0382] However, the content supply system ex100 is 
not limited to the combination as shown in FIG. 35, and 
may be connected to a combination of any of them. Also, 
each of the devices may be connected directly to the 

'5 telephone network exi 04, not via the base stations 
exi 07 ex110, which are fixed wireless stations. 
[0383] The camera exi 1 3 is a device such as a digital 
video camera capable of shooting moving pictures. The 
cellular phone may be a cellular phone of a PDC (Per- 

20 sonal Digital Communication) system, a CDMA (Code 
Division Multiple Access) system, a W-CDMA (Wide- 
band-Code Division Multiple Access) system or a GSM 
(Global System for Mobile Communications) system, a 
PHS (Personal Handyphone system) or the like, and 

25 may be any one of these. 

[0384] Furthermore, a streaming server ex1 03 is con- 
nected to the camera ex113 via the base station exi 09 
and the telephone network exi 04, which enables live 
distribution or the like based on coded data transmitted 

30 by the user using the camera exi 1 3. Either the camera 
ex1 1 3 or a server and the like capable of data transmis- 
sion processing may code the shot data. Also, moving 
picture data shot by a camera ex1 1 6 may be transmitted 
to the streaming server exi 03 via the computer ex111 . 

35 The camera exi 1 6 is a device such as a digital camera 
capable of shooting still pictures and moving pictures. 
In this case, either the camera exi 16 or the computer 
ex111 may code the moving picture data. An LSI exi 17 
included in the computer exi 11 or the camera exi 16 

40 perfomns coding processing . Note that software for cod- 
ing and decoding pictures may be integrated into a cer- 
tain type of storage medium (such as a CD-ROM, a flex- 
ible disk and a hard disk) that is a recording medium 
readable by the computer exi 11 and the like. Further- 

45 more, the camera-equipped cellular phone exi 15 may 
transmit the moving picture data. This moving picture 
data is data coded by an LSI included in the cellular 
phone ex115. 

[0385] In the content supply system ex100, content 
50 (e.g. a music live video) which has been shot by the user 
using the camera exi 13, the camera ex116 or the tike 
is coded in the same manner as the above-described 
embodiments and transmitted to the streaming server 
ex103, and the streaming server ex 103 makes stream 
55 distribution of the content data to clients at their request. 
The clients here include the computer exi 11, the PDA 
exi 12. the camera exi 13, the cellular phone exi 14 and 
so forth capable of decoding the above coded data. The 
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content supply system exi 00 with the above conf igura- 
tion is a system that enables the clients to receive and 
reproduce the coded data and realizes personal broad- 
casting by allowing them to receive, decode and repro- 
duce the data in real time. 

[0386] The picture coding apparatus and the picture 
decoding apparatus presented in the above embodi- 
ments can be used for coding and decoding to be per- 
fomried in each of the devices making up the above sys- 
tem. 

[0387] An explanation is given of a cellular phone as 
an example. 

[0388] FIG. 36 is a diagram showing the cellular 
phone ex115 that employs the picture coding method 
and the picture decoding method explained in the above 
embodiments. The cellular phone exil 5 has an antenna 
ex201 for transmitting/receiving radio waves to and from 
the base station ex110, a camera unit ex203 such as a 
CCD camera capable of shooting video and still pic> 
tures, a display unit ex202 such as a liquid crystal dis- 
play for displaying the data obtained by decoding video 
and the like shot by the camera unit ex203 and video 
and the like received by the antenna ex201 , a main body 
including a group of operation keys ex204, a voice out- 
put unit ex208 such as a speaker for outputting voices, 
a voice Input unit ex205 such as a microphone for input- 
ting voices, a recording medium ex207 for storing coded 
data or decoded data such as data of moving or still pic- 
tures shot by the camera, data of received e-mails and 
moving picture data or still picture data, and a slot unit 
ex206 for enabling the recording medium ex207 to be 
attached to the cellular phone exi 1 5. The recording me- 
dium ex207 is embodied as a flash memory element, a 
kind of EE PROM (Electrically Erasable and Program- 
mable Read Only Memory) that is an electrically erasa- 
ble and rewritable nonvolatile memory, stored in a plas- 
tic case such as an SD card. 

[0389] Next, referring to FIG. 37, an explanation is giv- 
en of the cellular phone ex115. In the cellular phone 
ex11 5, a main control unit ex311 for centrally controlling 
the display unit ex202 and each unit of the main body 
having the operation keys ex204 is configured in a man- 
ner in which a power supply circuit unit ex31 0, an oper- 
ation input control unit ex304, a picture coding unit 
ex312, a camera interface unit ex303, an LCD (Liquid 
Crystal Display) control unit ex302, a picture decoding 
unit ex309, a multiplexing/demultiplexing unit ex30B, a 
recording/reproducing unit ex307, a modem circuit unit 
ex306, and a voice processing unit ex305 are intercon- 
nected via a synchronous bus ex313. 
[0390] When a call-end key or a power key is turned 
on by the user's operation, the power supply circuit unit 
ex310 supplies each unit with power from a battery 
pack, so as to activate the camera-equipped digital cel- 
lular phone ex115 to make it into a ready state. 
[0391] in the cellular phone exi 15, the voice process- 
ing unit ex305 converts a voice signal received by the 
voice Input unit ex205 In conversation mode into digital 



voice data under the control of the main control unit 
ex311 comprised of a CPU, a ROM, a RAM and others, 
the modem circuit unit ex306 perfomis spread spectrum 
processing on it, and a transmit/receive circuit unit 

5 ex301 performs digital-to-analog conversion processing 
and frequency transformation processing on the data, 
so as to transmit the resultant via the antenna ex201 . 
Also, in the cellular phone ex115, data received by the 
antenna ex201 in conversation mode is amplified and 

10 performed of frequency transformation processing and 
analog-to-dlgital conversion processing, the modem cir- 
cuit unit ex306 perfomris inverse spread spectrum 
processing on the resultant, and the voice processing 
unit ex305 converts it into analog voice data, so as to 

IS output it via the voice output unit ex208. 

[0392] Furthemriore, when sending an e-mail in data 
communication mode, text data of the e-mail inputted 
by operating the operation keys ex204 on the main body 
is sent out to the main control unit ex311 via the opera- 
te tion input control unit ex304. In the main control unit 
ex311, after the modem circuit unit ex306 performs 
spread spectrum processing on the text data and the 
transmit/receive circuit unit ex301 performs digital-to- 
analog conversion processing and frequency transfer- 

2s mation processing on it, the resultant is transmitted to 
the base station exil 0 via the antenna ex201 . 
[0393] When picture data is transmitted in data com- 
munication mode, the picture data shot by the camera 
unit ex203 is supplied to the picture coding unit ex312 

30 via the camera Interface unit ex303. When picture data 
Is not to be transmitted, it is also possible to display such 
picture data shot by the camera unit ex203 directly on 
the display unit ex202 via the camera interface unit 
ex303 and the LCD control unit ex302. 

35 [0394] The picture coding unit ex312, which includes 
the picture coding apparatus according to the present 
invention in its configuration, perfonns compression 
coding on the picture data supplied from the camera unit 
ex203 using the coding method used by the picture cod- 

40 ing apparatus presented in the above-mentioned em- 
bodiments, so as to convert it into coded picture data, 
and sends it out to the multiplexing/demultiplexing unit 
ex308. At this time, the cellular phone exil 5 sends voic- 
es received by the voice Input unit ex205 while the 

45 shooting by the camera unit ex203 is taking place, to the 
multiplexing/demultiplexing unit ex308 as digital voice 
data via the voice processing unit ex305. 
[0395] The multiplexing/demultiplexing unit ex308 
multiplexes the coded picture data supplied from the pic- 

50 ture coding unit ex31 2 and the voice data supplied from 
the voice processing unit ex305 using a predetermined 
method, the modem circuit unit ex306 performs spread 
spectrum processing on the resulting multiplexed data, 
and the transmit/receive circuit unit ex301 performs dig- 

55 ital-to-analog conversion processing and frequency 
transformation processing on the resultant, so as to 
transmit the processed data via the antenna ex201 . 
[0398] When receiving, in data communication mode. 
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data included in a moving picture file which is linked to 
a Web page or the like, the nnodem circuit unit ex306 
perfonms inverse spread spectrum processing on the re- 
ceived data received from the base station ex1 1 0 via the 
antenna ex201 , and sends out the resulting multiplexed 
data to the multiplexing/demultiplexing unit ex308. 
[0397] In order to decode the multiplexed data re- 
ceived via the antenna ex201 , the multiplexing/demulti- 
plexing unit ex308 separates the multiplexed data into 
a bit stream of picture data and a bit stream of voice 
data, and supplies such coded picture data to the picture 
decoding unit ex309 and such voice data to the voice 
processing unit ex305 via the synchronous bus ex313. 
[0398] Next, the picture decoding unit ex309, which 
includes the picture decoding apparatus according to 
the present invention in its configuration, decodes the 
bit stream of the picture data using the decoding method 
paired with the coding method shown in the above-men- 
tioned embodiments 'so as to generate moving picture 
data for reproduction, and supplies such data to the dis- 
play unit ex202 via the LCD control unit ex302. Accord- 
ingly, moving picture data included in the moving picture 
file linked to a Web page, for instance, is displayed. At 
the same time, the voice processing unit ex305 converts 
the voice data into analog voice data, and then supplies 
this to the voice output unit ex208. Accordingly, voice 
data included in the moving picture file linked to a Web 
page, for instance, is reproduced. 
[0399] Note that the aforementioned system is not an 
exclusive example and therefore that at least either the 
picture coding apparatus or the picture decoding appa- 
ratus of the above embodiments can be incorporated 
into a digital broadcasting system as shown in FIG. 38, 
against the backdrop that satellite/terrestrial digital 
broadcasting has been a recent topic of conversation. 
To be more specific, at a broadcasting station ex409, a 
bit stream of video Infonnation is transmitted, by radio 
waves, to a satellite ex41 0 for communications or broad- 
casting. Upon receipt of it, the broadcast satellite ex410 
transmits radio waves for broadcasting, an antenna 
ex406 of a house equipped with satellite broadcasting 
reception facilities receives such radio waves, and an 
apparatus such as a television (receiver) ex401 and a 
set top box (STP) ex407 decodes the bit stream and re- 
produces the decoded data. The picture decoding ap- 
paratus as shown in the above-mentioned embodi- 
ments can be implemented in the reproduction appara- 
tus ex403 for reading and decoding the bit stream re- 
corded on a storage medium ex402 that is a recording 
medium such as a CD and a DVD. In this case, a repro- 
duced video signal is displayed on a monitor ex404. It 
is also conceived that the picture decoding apparatus is 
implemented in the set top box ex407 connected to a 
cable ex405 for cable television or the antenna ex4Q6 
for satellite/terrestrial broadcasting so as to reproduce 
it on a television monitor ex408. In this case, the pk:ture 
decoding apparatus may be incorporated into the tele- 
vision, not In the set top box. Or, a car ex412 with an 



antenna ex411 can receive a signal from the satellite 
ex410, the base station ex107 or the like, so as to re- 
produce a moving pteture on a display device such as 
a car navigation system ex413 mounted on the car 
5 ex412. 

[0400] Furthermore, it is also possible to code a pic- 
ture signal by the picture coding apparatus presented in 
the above embodiments and to record the resultant in a 
recording medium. Examples include a DVD recorder 
for recording a picture signal on a DVD disc ex421 and 
a recorder ex420 such as a disc recorder for recording 
a picture signal on a hard disk. Moreover, a picture sig- 
nal can also be recorded in an SD card ex422. If the 
recorder ex420 is equipped with the picture decoding 
apparatus presented In the above embodiments, It is 
possible to reproduce a picture signal recorded on the 
DVD disc ex421 or in the SD card ex422, and display it 
on the monitor ex408. 

[0401] As the configuration of the car navigation sys- 
tem ex413, the configuration without the camera unit 
ex203 and the camera Interface unit ex303, out of the 
configuration shown in FIG. 37, is conceivable. The 
same is applicable to the computer ex1 1 1 , the television 
(receiver) ex401 and the like. 

[0402] Concerning the terminals such as the cellular 
phone ex114, a transmitting/receiving terminal having 
both an encoder and a decoder, as well as a transmitting 
terminal only with an encoder, and a receiving terminal 
only with a decoder are possible as forms of implemen- 
tation. 

[0403] As stated above, it is possible to employ the 
picture coding method and the picture decoding method 
presented in the above embodiments into any one of the 

above-described devices and systems. Accordingly, it 
becomes possible to achieve an effect described in the 
aforementioned embodiments. 

[0404] From the invention thus described, it will be ob- 
vious that the embodiments of the invention may be var- 
ied in many ways. Such variations are not to be regarded 
as a departure from the spirit and scope of the invention, 
and all such modifications as would be obvious to one 
skilled in the art are intended for inclusion within the 
scope of the following claims. 

Industrial Applicability 

[0405] The picture coding method and the picture de- 
coding method according to the present invention is suit- 
ed for use in a picture coding apparatus for coding a 
moving picture and a picture decoding apparatus for de- 
coding the coded moving picture, as well as in a system 
equipped with these apparatuses such as a content sup- 
ply system for supplying content like digital wori< and a 
digital broadcast system. 
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Claims 

1 . A picture coding method for coding a current picture 
to be coded on a block-by-block basis with refer- 
ence to already coded pictures, connprising: 

a second reference picture selection step of se- 
lecting, as a second reference picture, anyone 
of coded pictures located after the current pic- 
ture in display order; 

a first reference picture specification step of 
specifying, as a first reference picture, any one 
of coded pictures based on a motion vector 
used for coding a predetennined block in the 
second reference picture; 
a predictive image generation step of specify- 
ing a block in the first reference picture and a 
block in the second reference picture based on 
the motion vector, and generating a predictive 
image from said two blocks through pixel inter- 
polation; 

a coding step of coding a difference between 
the predictive image and an image of a current 
block to be coded; and 

an output step of outputting specification infor- 
mation for specifying the second reference pic- 
ture, together with a result of the coding per- 
fomied in the coding step. 

2. The picture coding method according to Claim 1 , 

wherein in the output step, the specification 
infomriation that includes a second reference index 
indicating the second reference picture is outputted. 

3. The picture coding method according to Claim 1 , 

wherein in the output step, information indi- 
cating a method for changing second reference In- 
dices is outputted as the specification information, 
said second reference indices being assigned to the 
respective coded pictures in order to identify said 
coded pictures. 

4. A picture coding method for coding a current picture 
to be coded on a block-by-block basis with refer- 
ence to already coded pictures, comprising: 

a second reference picture selection step of se- 
lecting, as a second reference picture, a coded 
picture from among coded pictures that are lo- 
cated after the current picture in display order, 
said coded picture being assigned a smallest 
second reference index of all second reference 
indices assigned to the respective coded pic- 
tures in order to identify said coded pictures; 
a first reference picture specification step of 
specifying, as a first reference picture, any one 
of coded pictures based on a motion vector 
used for coding a predetermined block in the 



second reference picture; 
a predictive image generation step of specify- 
ing a block in the first reference picture and a 
block in the second reference picture based on 
s the motion vector, and generating a predictive 

image from said two blocks through pixel inter- 
polation; and 

a coding step of coding a difference between 
the predictive image and an image of a current 
10 block to be coded. 

5. The picture coding method according to Claim 4, 

wherein in the second reference picture se- 
lection step, the coded picture with the smallest sec- 
is ond reference index is specified according to as- 
signment infomnation indicating how the second ref- 
erence indices are assigned, and said specified 
coded picture Is selected as the second reference 
picture. 

20 

6. A picture coding method for coding a current picture 
to be coded on a block-by-block basis with refer- 
ence to already coded pictures, comprising: 

25 an assignment step of assigning, to respective 

coded pictures, second reference indices for 
identifying said coded pictures in a manner in 
which a second reference index indicating 0 is 
assigned to any one of coded pictures located 

30 after the current picture in display order; 

a re-assignment step, in which it is possible to 
re-assign the second reference indices in a 
manner different from the manner used in the 
assignment step, of assigning the second ref- 

55 erence index indicating 0 only to any one of the 

coded pictures located after the current picture 
in display order; 

a second reference picture selection step of se- 
lecting, as a second reference picture, the cod- 

^0 ed picture assigned the second reference index 

indicating 0 from among the coded pictures lo- 
cated after the current picture in display order; 
a first reference picture specification step of 
specifying, as a first reference picture, any one 

^5 of the coded pictures based on a motion vector 

used for coding a predetermined block in the 
second reference picture; 
a predictive image generation step of specify- 
ing a block in the first reference picture and a 

so block in the second reference picture based on 

the motion vector, and generating a predictive 
image from said two blocks through pixel inter- 
polation; and 

a coding step of coding a difference between 
5s the predictive image and an image of a current 

block to be coded. 

7. The picture coding method according to Claim 6, 
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wherein in the re-assignment step, it is pro- 
hibited to change the second reference index of the 
coded picture which is assigned the second refer- 
ence index indicating 0 in the assignment step. 

5 

8. A picture coding method for coding a current picture 
to be coded on a block-by-block basis with refer- 
ence to already coded pictures, comprising: 

an assignment step of assigning, to respective io 
coded pictures, first reference indices for iden- 
tifying said coded pictures; 
a re-assignment step of re-assigning the first 
reference indices to predetenmined coded pic- 
tures or slices in a manner different from the is 
manner used in the assignment step; 
a first reference picture selection step of select- 
ing, as a first reference picture, a coded picture 
assigned a first reference index indicating a 
predetennined value, from among coded pic- 20 
tures located before the current picture in dis- 
play order; 

a predictive image generation step of generat- 
ing a predictive image through pixel interpola- 
tion based on a predetermined block in the first 25 
reference picture; and 

a coding step of coding a difference between 
the predictive image and an image of a current 
block to be coded, 

30 

wherein in the re-assignment step, the first 
reference Index indicating said predetermined val- 
ue is assigned preferentially to the coded picture 
which is referred to for a long period of time. 

35 

9, A picture coding method for coding a current picture 
to be coded on a block-by-block basis with refer- 
ence to already coded pictures, comprising: 

an assignment step of assigning, to respective 40 
coded pictures, second reference indices for 
identifying said coded pictures; 
a second reference picture selection step of se- 
lecting, as a second reference picture, a coded 
picture which is assigned a second reference 
index indicating 0, from among coded pictures 
located after the current picture In display or- 
der; 

a first reference picture specifteation step of 
specifying, as a first reference picture, any one so 
of the coded pictures based on a motion vector 
used for coding a predetennined block In the 
second reference picture; 
a predictive image generation step of specify- 
ing a block in the first reference picture and a ss 
block in the second reference picture based on 
the motion vector, and generating a predictive 
image from said two blocks through pixel inter- 
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polation; and 

a coding step of coding a difference between 
the predictive image and an image of a current 
block to be coded, 

wherein in the assignment step, the second 
reference Index Indicating 0 is prohibited from being 
assigned to a coded picture that has been located 
before the current picture in display order. 

10. The picture coding method according to Claim 9, 
further comprising a deletion step of deleting, from 
a memory storing motion vectors used for coding 
the coded pictures, a motion vector used for coding 
the coded picture that has been located before the 
current picture in display order. 

11 . A picture coding method for coding a current picture 
to be' coded on a biock-by-block basis with refer- 
ence to already coded pictures, comprising: 

a selection step of selecting, from among a plu- 
rality of coded pictures stored in a storage unit, 
a coded picture corresponding to a first refer- 
ence index as a first reference picture and a 
coded picture corresponding to a second refer- 
ence index as a second reference picture, by 
use of said first reference index and said sec- 
ond reference index assigned to said plurality 
of coded pictures; 

a predictive image generation step of generat- 
ing a predictive Image from a block in the first 
reference picture and a block in the second ref- 
erence picture through pixel interpolation; and 
an output step of coding a prediction error 
which is a difference between the current pic- 
ture and the predictive image, and outputting a 
coded picture signal that includes a signal ob- 
tained by coding said prediction error, 

wherein in the selection step, the coded pic- 
ture which satisfies the following condition is select- 
ed as the first reference picture from among the 
coded pictures: a coded picture which is located be- 
fore the current picture in display order and which 
is assigned a smallest reference index, and 

in the predictive image generation step, ( i ) a 
second motion vector of a block which refers to the 
first reference picture is calculated, based on a first 
motion vector used to code said block which is In 
the second reference picture and co-located with a 
current block to be coded in the current picture, ( ii ) 
a third motion vector of the current block which re- 
fers to the first reference picture is cateulated and 
a fourth motion vector of the current block which re- 
fers to the second reference picture is calculated, 
both based on said second motion vector, and ( iii ) 
the predictive image is generated through pixel in- 
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terpoiation from a block in the first reference picture 
referred to by the third nnotion vector and a block in 
the second reference picture referred to by the 
fourth motion vector. 

12. A picture coding method for coding a current picture 
to be coded on a block-by-block basis with refer- 
ence to already coded pictures, comprising: 

a selection step of selecting, from among a plu- 
rality of coded pictures stored in a storage unit, 
a coded picture corresponding to a first refer- 
ence index as a first reference picture and a 
coded picture corresponding to a second refer- 
ence index as a second reference picture, by 
use of said first reference index and said sec- 
ond reference index assigned to said plurality 
of coded pictures; 

a predictive image generation step of generat- 
ing a predictive image from a block in the first 
reference picture and a block in the second ref- 
erence picture through pixel interpolation; and 
an output step of coding a prediction en-or 
which is a difference between the current pic- 
ture and the predictive image, and outputting a 
coded picture signal that includes a signal ob- 
tained by coding said prediction en-or, 

wherein in the selection step, the coded pic- 
ture located before the current picture in display or- 
der is selected as the first reference picture from 
among the coded pictures, 

in the predictive image generation step, ( i ) a 
second motion vector of a block which refers to the 
first reference picture Is calculated, based on a first 
motion vector used to code said block which is in 
the second reference picture and co-located with a 
current block to be coded in the current picture, ( ii ) 
a third motion vector of the current block which re- 
fers to the first reference picture Is calculated and 
a fourth motion vector of the current block which re- 
fers to the second reference picture is calculated, 
both based on said second motion vector, and ( iii ) 
the predictive image is generated through pixel in- 
terpolation from a block in the first reference picture 
refen-ed to by the third motion vector and a block in 
the second reference picture referred to by the 
fourth motion vector, and 

in the output step, the first reference index in- 
dicating the first reference picture selected in the 
selection step is incorporated into the coded picture 
signal. 

13. The picture coding method according to Claim 12, 
further comprising a step of prohibiting information 
from being stored into a memory, said information 
indicating a coded picture refenred to by the first mo- 
tion vector. 



14. A picture decoding method for decoding a coded 
picture on a block-by-block basis with reference to 
already decoded pictures, comprising: 

5 an obtainment step of obtaining specification 

infomnation and a coded signal, the specifica- 
tion information being used for specifying, as a 
second reference picture, a decoded picture to 
be referred to for decoding a current picture to 

^0 be decoded and the coded signal being related 

to said current picture; 

a second reference picture selection step of se- 
lecting the second reference picture from 
among decoded pictures that are located after 
IS the current picture in display order based on the 

specification information; 
a first reference picture specification step of 
specifying, as a first reference picture, any one 
of decoded pictures based on a motion vector 
used for decoding a predetermined block in the 
second reference picture; 
a predictive image generation step of specify- 
ing a block in the first reference picture and a 
block in the second reference picture based on 
the motion vector, and generating a predictive 
image from said two blocks through pixel Inter- 
polation; and 

a decoding step of decoding a current block to 
be decoded based on the predictive Image and 
30 the coded signal. 

15. The picture decoding method according to Claim 
14, 

wherein in the obtainment step, the specifica- 
35 tion information that includes a second reference in- 
dex indicating the second reference picture Is ob- 
tained, and 

in the second reference picture selection step, 
the second reference picture indk;ated by said sec- 
40 ond reference index is selected. 

16. The picture decoding method according to Claim 
14. 

wherein in the obtainment step, the specifica- 
45 tion information that indicates a method for chang- 
ing second reference indices is obtained, said sec- 
ond reference Indices being assigned to the respec- 
tive decoded pictures in order to identify said de- 
coded pictures, and 
50 in the second reference picture selection step, 

the decoded picture indicated by said method for 
changing the second reference indices is selected 
as the second reference picture. 

55 17. A picture decoding method for decoding a coded 
picture on a block-by-block basis with reference to 
already decoded pictures, comprising: 
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an obtainment step of obtaining a coded signal 
related to a cuaent picture to be decoded; 
a second reference picture selection step of se- 
lecting, as a second reference picture, a decod- 
ed picture from among decoded pictures that 5 
are located after the current picture In display 
order, said decoded picture being assigned a 
smallest second reference index of all second 
reference indices assigned to the respective 
decoded pictures in order to identify said de- io 
coded pictures,; 

a first reference picture specification step of 
specifying, as a first reference picture, any one 
of the decoded pictures based on a motion vec- 
tor used for decoding a predetermined block in is 
the second reference picture; 
a predictive image generation step of specify- 
ing a block in the first reference picture and a 
block in the second reference picture based on 
the motion vector, and generating a predictive 20 
image from said two blocks through pixel inter- 
polation; and 

a decoding step of decoding a current block to 
be decoded based on the predtetive image and 
the coded signal. 25 

18. The picture decoding method according to Claim 
17. 

wherein In the second reference picture se- 
lection step, the decoded picture with the smallest 30 
second reference index Is specified according to as- 
signment information indicating how the second ref- 
erence indices are assigned, and said specified de- 
coded picture is selected as the second reference 
picture. 35 

19. A picture decoding method for decoding a coded 
picture on a block-by-block basis with reference to 
already decoded pictures, comprising: 

40 

an obtainment step of obtaining a coded signal 
related to a current picture to be decoded; 
a second reference picture selection step of se- 
lecting, as a second reference picture, a decod- 
ed picture with a second reference Index of 0, 45 
from among decoded pictures which are locat- 
ed after the cunrent picture in display order and 
which are assigned second reference indices 
in order to Identify said respective decoded pic- 
tures; 50 
a first reference picture specification step of 
specifying, as a first reference picture, any one 
of decoded pictures based on a motion vector 
used for decoding a predetenmined block In the 
second reference picture; 55 
a predictive image generation step of specify- 
ing a block In the first reference picture and a 
block In the second reference picture based on 



the motion vector, and generating a predictive 
image from said two blocks through pixel inter- 
polation; 

a decoding step of decoding a current block to 
be decoded based on the predictive image and 
the coded signal, and 

a deletion step of deleting, from a memory stor- 
ing motion vectors used for decoding the de- 
coded pictures, a motion vector used for decod- 
ing a decoded picture that has been located be- 
fore the current picture in display order. 

20. A picture decoding method for decoding a coded 
picture on a block-by-block basis with reference to 
already decoded pictures, comprising: 

an obtainment step of obtaining a coded signal 
indicating a prediction error; 
a selection step of selecting, from among a plu- 
rality of decoded pictures stored in a storage 
unit, a decoded picture corresponding to a first 
reference index as a first reference picture and 
a decoded picture corresponding to a second 
reference index as a second reference picture, 
by use of said first reference index and said 
second reference index assigned to said plural- 
ity of decoded pictures; 

a predictive Image generation step of generat- 
ing a predictive Image from a block In the first 
reference picture and a block in the second ref- 
erence picture through pixel interpolation; 
a decoding step of decoding the current picture 
to be decoded according to the predictive im- 
age and the prediction error; and 
a storage step of storing, in the storage unit, a 
decoded picture with a possibility to be used as 
a reference picture, 

wherein in the selection step, the decoded 
picture which satisfies the following condition is se- 
lected as the first reference picture from among the 
decoded pictures: a decoded picture which is locat- 
ed before the current picture In display order and 
which is assigned a smallest first reference index, 
and 

In the predk:tlve image generation step. ( i ) a 
second motion vector of a block which refers to the 
first reference picture is calculated, based on a first 
motion vector used to decode said block which is In 
the second reference picture and co-located with a 
current block to be decoded in the current picture, 
( 11 ) a third motion vector of the current block which 
refers to the first reference picture is calculated and 
a fourth motion vector of the current block which re- 
fers to the second reference picture is calculated, 
both based on said second motion vector, and (iii) 
the predictive image is generated through pixel in- 
terpolation from a block In the tirst reference picture 
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referred to by the third motion vector and a blocl< in 
the second reference picture refen^ed to by the 
fourth motion vector. 

21. A picture decoding method for decoding a coded 
picture on a blocl<-by-block basis with reference to 
already decoded pictures, comprising: 

an obtainment step of obtaining a coded picture 
signal that includes a coded picture signal indi- 
cating a prediction error; 
a selection step of selecting, from among a plu- 
rality of decoded pictures stored in a storage 
unit, a decoded picture corresponding to a first 
reference index as a first reference picture and 
a decoded picture corresponding to a second 
reference index as a second reference picture, 
by use of said first reference index and said 
second reference index assigned to said plural- 
ity of decoded pictures; 

a predictive Image generation step of generat- 
ing a predictive Image from a block in the first 
reference picture and a block in the second ref- 
erence picture through pixel Interpolation; 
a decoding step of decoding the current picture 
to be decoded according to the predictive im- 
age and the prediction error; and 
a storage step of storing, in the storage unit, a 
decoded picture with a possibility to be used as 
a reference picture, 

wherein in the selection step, the decoded 
picture indicated by the first reference index includ- 
ed in the coded picture signal is selected as the first 
reference picture from among the decoded pfc- 
tures:, and 

in the predictive Image generation step, { i ) a 
second motion vector of a block which refers to the 
first reference picture is calculated, based on a first 
motion vector used to decode said block which is in 
the second reference picture and co-located with a 
current block to be decoded in the current picture, 
( ii ) a third motion vector of the current block which 
refers to the first reference picture is calculated and 
a fourth motion vector of the current block which re- 
fers to the second reference picture is calculated, 
both based on said second motion vector, and ( ill ) 
the predictive image is generated through pixel in- 
terpolation from a block in the first reference picture 
refen-ed to by the third motion vector and a block in 
the second reference picture referred to by the 
fourth motion vector. 

22. A picture coding apparatus for coding a current pic- 
ture to be coded on a btock-by-block basis with ref- 
erence to already coded pictures, comprising: 

a second reference picture selection unit oper- 



able to select, as a second reference picture, 
any one of coded pictures located after the cur- 
rent picture in display order; 
a first reference picture specification unit oper- 

5 able to specify, as a first reference picture, any 

one of coded pictures based on a motion vector 
used for coding a predetermined block in the 
second reference picture; 
a predictive image generation unit operable to 

10 specify a block In the first reference picture and 

a block in the second reference picture based 
on the motion vector, and generate a predictive 
image from said two blocks through pixel inter- 
polation; 

a coding unit operable to code a difference be- 
tween the predictive image and an image of a 
current block to be coded; and 
an output unit operable to output specification 
infomnation for specifying the second reference 
20 picture, together with a result of the coding per- 

formed by the coding unit. 

23. A picture decoding apparatus for decoding a coded 
picture on a block-by-block basis with reference to 

2s already decoded pictures, comprising: 

an obtainment unit operable to obtain specifi- 
cation information and a coded signal, the 
specification information being used for speci- 
30 fying, as a second reference picture, a decoded 

picture to be referred to for decoding a current 
picture to be decoded and the coded signal be- 
ing related to said current picture; 
a second reference picture selection unit oper- 
as able to select the second reference picture from 
among decoded pictures that are located after 
the current picture In display order based on the 
specification information; 
a first reference picture specification unit oper- 
40 able to specify, as a first reference picture, any 
one of decoded pictures based on a motion 
vector used for decoding a predetermined 
block In the second reference picture; 
a predictive image generation unit operable to 
45 specify.a block in the first reference picture and 
a block in the second reference picture based 
on the motion vector, and generate a predictive 
image from said two blocks through pixel Inter- 
polation; and 

^0 a decoding unit operable to decode a current 

block to be decoded based on the predictive im- 
age and the coded signal. 

24. A program for causing a computer to execute a pic- 
55 ture coding method for coding a current picture to 

be coded on a block-by-block basis with reference 
to already coded pictures, comprising: 
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a second reference picture selection step of se* 
iecting, as a second reference picture, any one 
of coded pictures located after the current pic- 
ture in display order; 

a first reference picture specification step of 5 
specifying, as a first reference picture, any one 
of coded pictures based on a motion vector 
used for coding a predetermined block in the 
second reference picture; 

a predictive image generation step of specify- io 
ing a block in the first reference picture and a 
block in the second reference picture based on 
the motion vector, and generating a predictive 
Image from said two blocks through pixel inter- 
polation; 15 
a coding step of coding a difference between 
the predictive image and an image of a current 
block to be coded; and 

an output step of outputting specification infor- 
mation for specifying the second reference pic- 20 
ture, together with a result of the coding per- 
fonned in the coding step. 

25. A program for causing a computer to execute a pic- 
ture decoding method for decoding a coded picture 25 
on a block-by-block basis with reference to already 
decoded pictures, comprising: 

an obtainment step of obtaining specification 
infonmation and a coded signal, the specif ica- 30 
tion information being used for specifying, as a 
second reference picture, a decoded picture to 
be referred to for decoding a current picture to 
be decoded and the coded signal being related 
to said current picture; 3s 
a second reference picture selection step of se- 
lecting the second reference picture from 
among decoded pictures that are located after 
the current picture in display order based on the 
specification Infomnation; 40 
a first reference picture specification step of 
specifying, as a first reference picture, any one 
of decoded pictures based on a motion vector 
used for decoding a predetemnlned block in the 
second reference picture; 45 
a predictive image generation step of specify- 
ing a block in the first reference picture and a 
block in the second reference picture based on 
the motion vector, and generating a predictive 
image from said two blocks through pixel inter- so 
polation; and 

a decoding step of decoding a current block to 
be decoded based on the predictive image and 
the coded signal. 
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FIG. 12 
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FIG. 16 
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FIG. 19 
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FIG. 21 
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FIG. 24 
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